API

Vision

Root image, codec, capture, media, viewer, and Vulkan Video values plus FnImage and FnDetection.

Compiler-extracted public headersNative C++ surface82 functions114 classes35 enums5 constants & variables
This inventory is extracted from OA public headers by clang-doc. Native types, pointers, references, defaults, public inheritance, comments and source ownership come from the compiler surface.

Examples

Vision

Functions

Classes

0 members

codec config parsed from av1C (AV1 ISO-BMFF §2.3): the configOBUs blob, which carries the sequence-header OBU out-of-band. AV1 MP4 samples are OBU temporal units that omit the sequence header, so we prepend this to every keyframe temporal unit (analogous to SPS/PPS for H.264/H.265).

Source
0 members

codec config parsed from avcC (H.264): SPS + PPS as Annex-B bytes, ready to prepend to the first IDR packet. Also exposes the NAL length field width so readNextPacket can rewrite length-prefix → start codes.

Source
15 members

Explicit completion and release boundary for producer-owned images and mapped ring slots still referenced by GPU consumers.

method

acquire and upload the newest available frame. Returns false when the camera has not produced another frame yet.

method

release a frame after its final consumer. The no-token overload declares immediate reuse; pass the exact completion when GPU work remains live.

method
Source
0 members
0 members
10 members
1 members
0 members

Encoded frame output — bitstream + PTS + key/keyflag.

Source
0 members

ISO-BMFF fragment defaults. `moov/mvex/trex` establishes this state; later `moof/traf/trun` boxes use it without retaining media payloads.

Source
0 members
0 members
0 members
oa::U32
oa::U32
Source
0 members
oa::U32
oa::U32
oa::U32
Source
0 members
0 members

codec config parsed from hvcC (H.265): VPS + SPS + PPS as Annex-B bytes

Source
0 members

Lightweight NAL unit descriptor — payload references back into the caller's bitstream buffer (no copy). For Annex-B framing the payload begins at the NAL header byte (skip the 00 00 00 01 / 00 00 01 prefix).

Source
0 members
1 members
0 members
oa::F32[3]
oa::F32[3]
Source
0 members

MP4 sample entry (used by parser)

Source
11 members

Stops the producer thread, completes exact frame-consumer dependencies, and releases portal, PipeWire, and vulkan resources.

method

Non-blocking. Returns true when a newer frame than the previous poll() is published through the common video-frame contract.

method

release a frame returned by poll(). The no-token overload declares that no asynchronous consumer remains.

method

GPU-deferred release. DMA-BUF ownership returns to PipeWire only after inConsumed completes. mapped fallback ring slots are likewise withheld from producer reuse until the exact completion becomes ready.

method
Source
0 members
1 members
0 members

Container information (probed from file header)

Source
0 members
0 members
36 members

hardware video decoder session. Wraps VkVideoSessionKHR + DPB management.

Explicit completion and resource-release boundary.

method

flush decoder state (call at stream end or seek).

method

Returns tightly packed two-plane 4:2:0 bytes in the frame's native storage representation: NV12 bytes or little-endian P010 16-bit words.

method
Source
21 members

Video stream demuxer

Av1Config
AvcConfig
FragmentConfig
HvcConfig
VideoContainerInfo
Vp9Config

Explicit resource-release boundary. Reports local file close failures; pending packets are discarded and no container data is manufactured.

method

Returns SPS+PPS concatenated as Annex-B (empty if not parsed). Caller prepends to the first IDR packet to bootstrap the decoder.

method

index of the next sample readNextPacket() will read (0-based). After decoding sample N, this equals N+1. Used by Video for step-back.

method

Return one indexed sample's presentation timestamp without converting a signed composition offset to unsigned first. Malformed tables whose offset underflows DTS or whose positive offset overflows fail closed.

method
Source
0 members

Network/live-source policy. local files ignore reconnect and jitter fields. Live transports use the same bounded packet/reconnect policy as file-backed demuxers. Protocol support is capability-gated by OA rather than inherited from a process-global third-party media registry.

Source
0 members
0 members

codec/profile-specific vulkan Video encode capabilities. Mirrors VideoDecodeCapabilities.

VkVideoEncodeCapabilityFlagsKHR
VkVideoEncodeH265CtbSizeFlagsKHR
VkVideoEncodeH265StdFlagsKHR
VkVideoEncodeH265TransformBlockSizeFlagsKHR
VkVideoEncodeRateControlModeFlagsKHR
Source
0 members
11 members

hardware video encoder session — wraps VkVideoSessionKHR for encoding.

Explicit completion and resource-release boundary. pending encoded output is discarded; call flush() first when the packets are required.

method

convert a buffer-backed RGBA texture and encode one frame through this session's owning engine. pending compute producers are completed before the encoder snapshots the source allocation.

method
Source
0 members

Timestamped frame shared by decode, capture, processing, presentation and encode. resource handles are non-owning and remain valid for the lifetime contract of their producer.

Source
4 members
12 members

Closes the output without writing a movie trailer. call finalize() first when a playable container is required.

method
method
Source
0 members
0 members
39 members

Explicit completion and resource-release boundary. Waits exact retained frame-consumer events before releasing decoder-owned image storage.

method

Most recently decoded frame. imageView is VK_NULL_HANDLE until the first successful decode.

method

convert the current decoder-owned frame to an ML tensor. conversion lives on VideoPlayer rather than VideoFrame because the decoder session owns the image views, YCbCr metadata, synchronization and conversion resources.

method

Unified viewer transport timebase. Public timeline APIs use microseconds; container-specific ticks remain private to VideoPlayer::seek.

method

drain outstanding GPU work and clear queued display frames.

method

Returns the number of frames decoded since the most recent rewind.

method

isDone() is permanently false in looping mode. in non-looping mode, flips to true once the underlying stream's EOS has been observed and the last decoded frame consumed.

method

Record the compute submission that most recently sampled currentFrame(). The session will not recycle that RGBA image until inConsumed completes. frames advanced without being rendered remain immediately reusable.

method

Advances by exactly one decoded frame (ignores playing_, ignores the wall-clock accumulator). Use tick() for time-paced playback. advance to the next display-order frame. open() presents the first frame, so the initial currentFrame() is immediately usable. next() reports decode, demux and synchronization failures instead of hiding them.

method

─── playback control ─────────────────────────────────────────────────

method

Read back the current converted RGBA frame. This is intended for CPU reference overlays and diagnostics; realtime inference should consume currentFrame() directly on the GPU.

method

Seek in the container video-track timebase. Ordinary seeks flush decoder state without destroying reusable RGBA allocations; genuine stream-format generation changes recreate the decoder.

method

Select an exact display-order frame. The caller owns play/pause policy; timestamp rounding remains confined to time-based seekUs().

method

Present the previous display-order frame. Cache-resident history does no decode work; a miss seeks to the preceding keyframe and rebuilds a bounded presentation window.

method

Scrub by a signed number of display-order frames. Positive movement advances normally; negative movement uses retained presentation history first and replays through the reorder path only on a cache miss.

method

advance by wall clock when playing. When paused, accumulates nothing (no implicit frame skipping after un-pause).

method

─── stream metadata passthroughs ─────────────────────────────────────

method
Source
0 members
0 members
0 members

exact stream profile for decoder capability queries and session creation.

Source
14 members

Releases encoder and host-file state without finalizing.

method

flush the encoder and finalize the container. idempotent.

method

Non-blocking image-input variant. outInputConsumed signals after the source image has returned to its published layout/queue family.

method
Source
0 members
3 members
0 members

Enums

Constant · Replicate · Reflect · Reflect101 · Wrap

Auto · Jpeg · Png · Webp · Bmp · Tga

Nearest · Bilinear · Bicubic

NV12 · RGB8 · RGBA8 · BF16 · F32

MonitorOrWindow · Monitor · Window

Bit8 · Bit10 · Bit12

Monochrome · Yuv420 · Yuv422 · Yuv444

H264 · H265 · AV1 · VP9

codec-standard profile. values are codec-qualified so a VideoProfile is self-describing without exposing vulkan std Video enums.

Unspecified · H264Baseline · H264Main · H264High · H264High444Predictive · H265Main · H265Main10 · H265MainStillPicture · H265FormatRangeExtensions · H265ScreenContentCodingExtensions · Av1Main · Av1High · Av1Professional · Vp9Profile0 · Vp9Profile1 · Vp9Profile2 · Vp9Profile3

Container kind enumeration

Unknown · Mp4 · WebM · MpegTs · Matroska

storage backing a video frame.

None · Image · Buffer

Progressive · InterlacedInterleavedLines · InterlacedSeparatePlanes

Decoded-frame resource path — selected once at decoder create() based on device capabilities + queue topology.

CoincidentFastStaging · DistinctComputeConvert · CoincidentComputeStaging · DirectCoincidentSampling · Unavailable

Auto · BT709 · BT2020

Functions

Pairwise IoU for FP32 center-x/center-y/width/height boxes. inA [N,4], inB [M,4] -> FP32 [N,M]. Coordinates may be normalized or pixel-valued, but widths and heights must be non-negative.

Multiclass semantic-segmentation metrics over flattened Int32 label matrices. labels outside [0,classCount) are ignored by the confusion accumulator, which provides the ordinary ignore-index behavior.

Deterministic class-aware NMS. boxes are FP32 [N,4] cx/cy/w/h, scores FP32 [N], and classes Int32 [N]. The implementation is GPU-resident and records as one graph node; it never sorts or compacts on the host.

Functions

Stateless still-image codec boundaries. Decode uploads one normalized Float32 NCHW image through the active engine. Encode and saveFile are explicit synchronous host boundaries which complete and read back their semantic image input.

Synchronous file sink for a device-local packed RGBA8 texture. pending work in the engine's matching private recorder completes before readback. The output codec is inferred from .png, .jpg/.jpeg, .bmp, or .tga.

Ui

Functions

Classes

0 members
oa::F32
oa::F32
oa::F32
oa::F32
oa::F32
Source
0 members
4 members

─── CvFrame ────────────────────────────────────────────────────────────────

const oavk::Buffer *
oa::I32
oa::I32
Source
0 members

─── CvOverlay ──────────────────────────────────────────────────────────────

CvBboxesConfig
Source
8 members
0 members
0 members
10 members
0 members
0 members

One source-anchored glyph quad. anchor is normalized to the source image; offsets and dimensions are compose-target pixels so labels remain readable while the image camera pans and zooms.

Source
11 members

Decode any stb_image-supported file to planar U8. 1-channel files → gray (channelCount=1). 3-channel files → RGB (channelCount=3). 4-channel files → RGBA (channelCount=4). HDR (.hdr) → planar F32. synchronous (transfer completes on return).

static

Records the exact UI consumer completion. Abandoning the value before completion transfers all owned plane buffers to engine retirement.

method
Source
11 members

─── InputSystem ────────────────────────────────────────────────────────────

YAML schema: {version: 1, bindings: [{action, key, modifiers, context, allow_repeat}]}. Loading is transactional for syntax and schema errors. Without yaml-cpp these calls report Unimplemented, never success.

method

register the reusable Realm tool bindings. This installs named actions with no callbacks; the application owns every operation they trigger: F12 — screenshot ctrl+R — start/stop recording Space — toggle camera ctrl+Z — undo (node graph) ctrl+Y — redo ctrl+S — save F — fit-to-view (node canvas)

method
Source
0 members

─── KeyAction ──────────────────────────────────────────────────────────────

Source
1 members

─── KeyBinding ─────────────────────────────────────────────────────────────

Source
24 members

Configuration and direct mutations are result-bearing. Rejected input never changes the current navigation state.

method
Source
0 members
0 members
14 members
const oa::F32
const oa::F32

── input: update pan/zoom from raw deltas ────────────────────────────────

method

── Coordinate transforms ─────────────────────────────────────────────────

method
Source
0 members
0 members
0 members

─── Caller-borrowed hit-test records ────────────────────────────────────────

WorldAabb
oa::U64
oa::I32
Source
0 members

─── NodeCanvas ─────────────────────────────────────────────────────────────

Source
3 members
0 members
0 members

─── TextLayout ───────────────────────────────────────────────────────────── CPU-side layout: strictly decodes UTF-8, selects explicit per-codepoint font fallback, and uses HarfBuzz for OpenType substitution, positioning, and source-byte clusters over OA's embedded left-to-right coverage. Paragraph BiDi and right-to-left script coverage are intentionally outside this API.

oa::U32
FontId
oa::F32
oa::F32
Source
2 members
8 members

─── TextAtlas ────────────────────────────────────────────────────────────── Manages the device-side coverage atlas and the glyph metadata table.

Source
2 members
0 members
0 members

Immutable generated artifact handed from a training/evaluation producer to the viewer. The texture allocation is shared; the timeline token prevents presentation until the producing GPU batch has completed. Replacing a pending frame drops it without ever blocking the training owner.

Source
0 members
13 members

Inherits public ViewerLiveSource

Source
74 members

─── Ui ──────────────────────────────────────────────────────────────────────

valid after widgets are submitted and until the next beginFrame. This is a flat, caller-borrowed platform-adapter snapshot, not an OA-owned desktop or document tree. bounds are ancestor-clipped physical pixels.

method

The viewport is required only by top-layer popup/tooltip placement. Viewer supplies its live compose extent. content scale converts the init style's logical geometry into physical pixels; event coordinates and explicit PixelRect values remain physical. Non-finite/non-positive scale is a frame error.

method

── layout containers ─────────────────────────────────────────────────────

method

beginPopup/endPopup render through the same deferred UI compositor, but their commands are submitted after every base-layer command. Popup calls must use the same parent scope as openPopup.

method

Explicit rows hug measured item widths and advance with layout.gap. Column items otherwise stretch across the panel's padded inner width.

method

Scroll panels retain only interaction/offset state under their stable ID. content coordinates are shifted and every nested draw/hit-test is clipped to the viewport. inContentHeight includes any layout padding. The caller owns content data; endScrollPanel must close the matching begin call.

method

Borrows the Viewer-owned atlas and allocates completion-tracked dynamic glyph slots. Bind once before the first frame; the atlas must outlive Ui.

method

labels identify controls within the active panel/row scope and must be unique there. mouse activation fires on release-inside; Tab/shift+Tab traverse focus and Return/Space activate the focused control. Returns true on click or keyboard activation.

method

Explicit renderer completion and release boundary. Waits every exact frame completion accepted by markFrameSubmitted before releasing pipelines, descriptor slots, and transient upload storage.

method

One reusable viewport/chart/canvas grid. The default decimal hierarchy is 10-unit minor, 100-unit major, 1000-unit super-major, plus black axes and a subtle theme-derived vertical background gradient.

method

Planar path: blitPlanar.slang handles per-channel dtype conversion + sRGB. inPlanes must remain at a stable address through markFrameSubmitted; that call attaches the exact consumer event used by its non-blocking RAII path.

method

Called once after init, before the first frame. inComposeImageView: vkImageView (as void*) of the compose storage image (set=1, slot 0).

method

── input state ───────────────────────────────────────────────────────────

method

Single-line UTF-8 editor. Returns true only when inOutText changes. The caller owns the value; the field retains bounded local edit history and IME composition state. It supports single/double/triple-click caret/word/all selection, shift and word navigation, scalar-safe deletion, clipboard, ctrl/Cmd+Z redo variants, Home/End, horizontal scrolling, visible native pre-edit and committed UTF-8 input. An external value replacement rebases local undo history; document-level undo remains application policy.

method

label does not soft-wrap; text wraps to the current panel's inner width. Both honor explicit newlines, require an active panel and a bound text atlas, and report misuse through recordRender instead of dropping text.

method

Marks transient resources sampled by this frame. Plot buffers are recycled only after the exact engine-owned graphics completion is reached.

method

at most one interactive popup owns pointer/keyboard input. The no-anchor overload anchors to the most recently submitted interactive item. The explicit overload supports context menus and application-defined anchors.

method

Finite fractions are clamped to [0, 1]. An empty overlay renders a rounded integer percentage; an explicit overlay is centered without soft wrapping.

method

── style stack (O(1), max depth 32) ─────────────────────────────────────

method

draw filled or outlined axis-aligned rectangles directly into the GPU compose image. Rectangles must already be clipped to its extent.

method

route a platform event. Returns true when an active/focused widget consumes it.

method

Draws a one-pixel horizontal rule in columns or a vertical rule in rows.

method

Sliders clamp the referenced value into [min, max] and return true only when it changes. Focused sliders accept arrow-key adjustment (1% of the range, shift=10x, ctrl=0.1x); pointer drags remain captured outside bounds.

method

Divides an explicit rectangle without beginning either child panel. The caller owns and may persist inOutRatio; it is the first region's fraction of the extent remaining after handleSize. Pointer drags retain capture outside the handle. Focused row splits accept Left/Right, and focused column splits accept Up/Down; shift/ctrl use the shared coarse/fine scale.

method

Draws one caller-owned tab sequence. Selection and overflow position live in inOutState; close and reorder are returned as requests so OA never owns documents or mutates their order. Left/Right/Home/End navigate the focused sequence and ctrl/Cmd+W requests closure of a closable focused tab.

method

Returns and clears a focused editor's pending copy/cut payload so the platform layer can publish it without introducing SDL into Ui.

method

Positioned GPU text primitive for plots, overlays, and other fixed-layout drawing. It does not participate in panel flow; inRect is an absolute compose-space clip/alignment rectangle. Stretch alignment is invalid.

method

Attaches to the immediately preceding interactive item. tooltip geometry is passive and shares the top-layer command route with popups.

method

One row of a caller-flattened hierarchy. The caller owns open/selection state and applies open when openChanged is returned. Pointer activation on the disclosure toggles only; activation elsewhere selects. Focused tree rows use Up/Down to select among prior rendered rows, Left/Right to close or open, and Return/Space to activate. Explicit rectangles make the row directly compatible with virtualRows.

method

Called after a compose image rebuild (resize) to refresh the image descriptor.

method

Returns the visible half-open row range for the active scroll panel. rows start after layout.padding.top and use inRowHeight + inRowGap stride. Overscan is expressed in rows and is clamped to the item count.

method

Platform bridge for enabling native committed-text/IME input only while a field rendered in the current frame owns keyboard focus.

method
Source
0 members
3 members
3 members

─── UiEvent ─────────────────────────────────────────────────────────────────

Source
0 members
0 members
1 members

─── UiInputState ──────────────────────────────────────────────────────────── Stable per-frame snapshot updated by Ui::beginFrame before widget dispatch.

Source
0 members
0 members
0 members

─── Widget config structs ────────────────────────────────────────────────────

Source
0 members
0 members
PixelRect
PixelRect
PixelRect
Source
0 members
0 members
0 members
0 members
3 members
0 members
0 members
5 members
0 members
0 members
0 members
0 members
0 members
0 members
0 members
0 members
1 members
24 members

Windowed or headless application session for inspecting OA images, video, audio and live GPU output. The Viewer owns UI, input, window and presenter state. It either creates one engine for run() or borrows the caller's engine for run(oa::Engine & ); it never creates a second device runtime in that path.

Unified blocking preview front door. Paths are decoder-probed as image, video, then audio and may either own an application engine or borrow the caller's presentation-capable engine. Direct GPU values always borrow the caller's engine and retain the exact completion contract of show().

static

run the windowed session against an existing presentation-capable engine. The viewer borrows the engine and closes only its presenter/window state.

method

Compatibility spelling for blocking one-shot display sinks. Matrix and image overloads record the RGBA8 conversion into the engine's matching private recorder, submit it once, and pass its exact completion into presentation. New code uses preview() so file-backed and direct values share one discoverable verb.

static

Image-backed renderer targets require their generation-safe frame and owning session. The blocking viewer returns its final graphics completion to that session before the presentation timelines are destroyed.

static
Source
0 members
0 members
10 members

Non-owning live producer attached to the one Viewer application lifecycle. It may render domain-specific overlays, but it never owns the window, swapchain, input pump or presentation submission.

Source
24 members

Viewport — Passive view description for rendering Declares: camera, target, viewport, and view-specific settings. Does NOT: handle input, own windows, manage swapchains, or animate.

═══════════════════════════════════════════════════════════════════════ clear values ═══════════════════════════════════════════════════════════════════════

method

═══════════════════════════════════════════════════════════════════════ layer filtering ═══════════════════════════════════════════════════════════════════════

method

═══════════════════════════════════════════════════════════════════════ Render target (required) ═══════════════════════════════════════════════════════════════════════

method

═══════════════════════════════════════════════════════════════════════ viewport and scissor ═══════════════════════════════════════════════════════════════════════

method
Source
2 members
3 members

─── WorldAabb ──────────────────────────────────────────────────────────────

Vec2
Vec2
Source

Enums

─── FontId ─────────────────────────────────────────────────────────────────

Sans · Mono

U8 · U16 · F32 · BF16

Global · NodeCanvas · TextInput · Timeline

═════════════════════════════════════════════════════════════════════════════ RenderLayer — layer/filter flags for selective rendering ═════════════════════════════════════════════════════════════════════════════

None · Default · Ui · Overlay · Background · Debug · All

None · Focus · Activate · Toggle · Increment · Decrement · SetValue · Close

─── Accessibility export ───────────────────────────────────────────────

Button · Checkbox · Slider · TextField · ComboBox · MenuItem · Tab · TreeItem · Splitter · Timeline

None · Disabled · Focused · Checked · Selected · Expanded · Editable · HasPopup

Start · Center · End · Stretch

─── layout primitives ────────────────────────────────────────────────────────

Column · Row

─── UiEventType ─────────────────────────────────────────────────────────────

None · MouseMove · MouseDown · MouseUp · MouseScroll · KeyDown · KeyUp · KeyChar · WindowResize · WindowClose · WindowFocus · WindowBlur · Pinch · TextEditing

─── UiKey ─────────────────────────────────────────────────────────────────── SDL3 scancode subset. values deliberately match SDL_Scancode to allow direct cast; do not rely on the numeric values in switch logic.

Unknown · A · B · C · D · E · F · G · H · I · J · K · L · M · N · O · P · Q · R · S · T · U · V · W · X · Y · Z · Num1 · Num2 · Num3 · Num4 · Num5 · Num6 · Num7 · Num8 · Num9 · Num0 · Return · Escape · Backspace · Tab · Space · Minus · Equals · Comma · Period · Slash · Home · Delete · End · Right · Left · Down · Up · F1 · F2 · F3 · F4 · F5 · F6 · F7 · F8 · F9 · F10 · F11 · F12 · KpEnter · Kp1 · Kp2 · Kp3 · Kp4 · Kp5 · Kp6 · Kp7 · Kp8 · Kp9 · Kp0

─── UiPinchPhase ────────────────────────────────────────────────────────────

None · Begin · Update · End

─── UiScrollGesture ─────────────────────────────────────────────────────── Classified once in the input layer (SDL wheel heuristics). Navigation maps each gesture to its own sensitivity — wheel zoom, touchpad pan, pinch zoom.

None · MouseWheel · TouchpadPan · PinchScroll

LeftToRight · BottomToTop

Waveform · Spectrum · Mel

Auto · Image · Video · Audio · Live

Image2D · Scene3D · Matrix · Video · CameraStream

Constants & Variables