Matrix multiplication

Build one GPU matrix product, validate it against an independent CPU reference, then measure representative shapes without confusing peak samples, replay throughput, or synchronized application latency.

TuCurrent contractC++ source-ownedCore

Contract	Value
Operation	`oa::FnMatrix::matMulNt(a, b)`
Shape	`A[M,K] · B[N,K]ᵀ → C[M,N]`
Work	`2 · M · N · K` floating-point operations
Routing	Internal selection from shape, precision, and live device capabilities
Current source	`sdk/cpp/tutorials/core/tuCoreMatMulIntro.cpp`
SDK filename target	`tuMatmul.cpp` / `tuMatmul.py`

1. Record one operation

OA stores B as [N,K] and computes A @ Bᵀ. The public call records the semantic operation; kernel selection remains an internal lowering decision. C++ submits explicitly. Python synchronizes when the result crosses the host boundary.

1#include <oa/oa.h>
2
3OA_MAIN("TutorialCoreMatmul") {
4	auto a = oa::FnMatrix::full({2, 3}, 1.0F);
5	auto b = oa::FnMatrix::full({2, 3}, 2.0F);
6	auto c = oa::FnMatrix::matMulNt(a, b);
7
8	auto submitted = engine.submit();
9	if (not submitted.isOk()) return 1;
10	if (not engine.wait(submitted.getValue()).isOk()) return 1;
11
12	oa::Array<oa::F32, 4> values{};
13	if (not oa::FnMatrix::copyToHost(c, values.data(), sizeof(values)).isOk()) return 1;
14	return 0;
15}
16

2. Prove correctness first

The tutorial checks tiny and regular squares, tall and wide projections, irregular dimensions, single-row decode, MNIST-like layers, and fused Linear bias. GPU output is compared with a separate FP32 CPU implementation before any performance row is valid.

1double maxError = 0.0;
2double maxReference = 1e-6;
3for (oa::Usize index = 0; index < reference.size(); ++index) {
4	maxError = oa::max(maxError,
5		oa::abs(static_cast<double>(reference[index]) - gpu[index]));
6	maxReference = oa::max(maxReference,
7		oa::abs(static_cast<double>(reference[index])));
8}
9const double normalizedError = maxError / maxReference;
10

The accepted normalized error is 1e-4 for FP32 and 3e-2 only on devices where the BF16 cooperative-matrix route is actually selected and verified.

3. Measure the route you mean

The primary result is median synchronized wall time. The 95th percentile exposes jitter and throttling. A fastest-of-N sample may describe an optimistic peak, but it is not representative throughput.

Section	What the number includes
Public operation	Allocation, recording, submission, GPU work, and synchronization
Pre-allocated output	The same route with output storage reused
Replay / pipeline ×8	An already captured graph submitted repeatedly before one wait
Batch ×4 / ×8	Independent GEMMs recorded into one submission, reported as batch and per-op time

For a batch of B independent products, aggregate throughput isB · (2·M·N·K) / batch wall time. The previous public page multiplied byB twice; those inflated batch figures are intentionally gone.

1oa::Matrix output;
2auto captured = engine.capture([&]() {
3	output = oa::FnMatrix::matMulNt(a, b);
4});
5if (not captured.isOk()) return 1;
6auto plan = oa::move(captured).getValue();
7
8auto event = engine.submit(plan);
9if (not event.isOk()) return 1;
10if (not engine.wait(event.getValue()).isOk()) return 1;
11

4. Current checked baseline

These are Release results recorded on 11 July 2026 with an Intel Iris Xe Graphics Tiger Lake GT2 device, Vulkan 1.4, and the verified FP32 fallback. They are historical evidence for this exact method—not a current cross-device performance claim.

Shape	M	N	K	p50 ms	p95 ms	GFLOP/s
square-512	512	512	512	0.709	0.877	378.8
square-1024	1024	1024	1024	4.901	5.967	438.2
square-2048	2048	2048	2048	41.615	47.522	412.8
tall-skinny	4096	128	1024	2.696	4.354	398.3
short-wide	128	4096	1024	2.690	2.778	399.1
gemv-decode	1	4096	4096	5.051	5.879	6.6

5. Read model shapes, not just squares

Large Alm projections approach the large-GEMM regime, while small tutorial-scale NLP projections remain dominated by dispatch and fixed costs. Single-token decode is GEMV-like and requires a dedicated route rather than another square-GEMM tile.

Shape	M	N	K	p50 ms	p95 ms	GFLOP/s
nlp-qkv	1024	32	32	0.076	0.253	27.4
nlp-ffn-up	1024	64	32	0.070	0.092	59.7
nlp-ffn-down	1024	32	64	0.078	0.088	53.8
alm-qkv	4096	384	384	2.911	3.789	414.9
alm-ffn-up	4096	1536	384	12.310	13.751	392.5
alm-ffn-down	4096	384	1536	13.346	18.112	362.0

6. Run and refresh the evidence

Build Release once, close competing GPU workloads, warm the process, then retain the complete recorded run with device, driver, precision, correctness, p50, and p95 output. When stronger hardware is available, replace this baseline from one controlled run rather than mixing new rows with the unavailable 5090 measurements.

cmake --build build/release --target TutorialCoreMatMulIntro -j
./bin/release/sdk/tutorials/core/tuCoreMatMulIntro

# Optional exhaustive CSV grid
OA_AUTOTUNE_BENCH=1 ./bin/release/sdk/tutorials/core/tuCoreMatMulIntro