Vision
Images, video, capture, and models on one GPU-first path.
OA Vision combines 50 schema-owned image operations, Vulkan Video decode and encode, timestamped capture, media containers, and direct oa::Matrix interoperability.
50 image operationsH.264 · H.265 · AV1 · VP9 decodeCapability-gated codecs
Vision
The frame contract
oa::VideoFrame carries the image or buffer resource, format, dimensions, presentation timestamp, color metadata, and GPU completion token. Decode, capture, processing, display, and encode use that same contract instead of copying through an intermediate CPU frame abstraction.
Image operations
The public inventory is generated from Vision schemas; complex graph lowering remains handwritten and tested.
Vulkan Video
Codec support and resource paths are selected from device capabilities, queue topology, formats, and profile limits.
Media sessions
oa::Video provides open, next, seek, flush, playback state, frame reordering, and optional synchronized audio.
ML interop
asMatrix and oa::FnImage preserve the direct path from decoded or captured imagery to model input.
Operation families
| Family | Coverage |
|---|---|
Pixel | Thresholding, range tests, clamp, inversion, tone controls, compositing, channel operations, and noise. |
Geometric | Resize, crop, flip, rotate, pad, remap, affine warp, and perspective warp. |
Filter | Convolution, blur, Sobel, Scharr, Laplacian, morphology, bilateral, sharpening, and adaptive thresholding. |
Color | Per-channel normalization plus semantic image color conversion. |
GPU augmentation pipeline
The semantic oa::Image contract and oa::FnImage operation names match across the checked C++ and Python tutorial sources.
1oa::Vector<oa::Image> views;2 views.reserve(6);3 views.pushBack(oa::move(decoded).getValue());4 const oa::Matrix& source = views.front().asMatrix();56 views.pushBack(rgbImage(oa::FnImage::flip(source, true, false)));7 auto crop = oa::FnImage::centerCrop(source, 280, 150);8 views.pushBack(rgbImage(oa::FnImage::resize(crop, 320, 180)));9 views.pushBack(rgbImage(10 oa::FnImage::brightnessContrast(source, 0.08F, 1.15F)));11 auto noisy = oa::FnImage::gaussianNoise(source, 0.0F, 0.035F, 2026U);12 views.pushBack(rgbImage(oa::FnImage::clamp(noisy, 0.0F, 1.0F)));13 views.pushBack(rgbImage(oa::FnImage::solarize(source, 0.55F, 1.0F)));14
Codec boundary
| Format | Route | Output contract |
|---|---|---|
| H.264 / AVC | Vulkan Video when supported | Timestamped NV12 or converted RGBA frame |
| H.265 / HEVC | Vulkan Video when supported | Timestamped NV12 or converted RGBA frame |
| AV1 | Vulkan Video when supported | Timestamped NV12 or converted RGBA frame |
| VP9 | Vulkan Video when supported | Timestamped NV12 or converted RGBA frame |
| JPEG / PNG | File decode and GPU upload | Texture, image planes, or oa::Matrix |
Support is queried at runtime. A listed codec is part of the OA implementation surface; it is not a claim that every Vulkan device exposes every profile, format, or queue route.