oa::FnMatrix::linearGelu

linearGelu: fused linear + GELU. output = GELU(input @ weight^T + bias) The fused forward discards the pre-activation; the autograd node recomputes it (one GEMM) for geluBwd in the backward pass.

FnC++ public headers

Function Documentation

Matrix oa::FnMatrix::linearGelu(
	const Matrix & inX,
	const Matrix & inWeight,
	const Matrix & inBias
)

linearGelu: fused linear + GELU. output = GELU(input @ weight^T + bias) The fused forward discards the pre-activation; the autograd node recomputes it (one GEMM) for geluBwd in the backward pass.

Parameters

inX

const Matrix &

—

inWeight

const Matrix &

—

inBias

const Matrix &

—

Returns

Matrix

The declared return value.