oa::FnMatrix::softmaxScaledMaskedBwd

softmaxScaledMaskedBwd: backward for the fused attention score op. Returns dScores = softmaxOut * (dOut - sum(dOut * softmaxOut)) * scale.

FnC++ public headers

Function Documentation

Matrix oa::FnMatrix::softmaxScaledMaskedBwd(
	const Matrix & inForwardOutput,
	const Matrix & inGradOutput,
	oa::F32 inScale
)

softmaxScaledMaskedBwd: backward for the fused attention score op. Returns dScores = softmaxOut * (dOut - sum(dOut * softmaxOut)) * scale.

inForwardOutput

const Matrix &

—

inGradOutput

const Matrix &

—

inScale

oa::F32

—

Matrix

The declared return value.