oa::FnMatrix::flashAttentionCausalBwd

flashAttentionCausalBwd: explicit adjoint for the FlashAttention autograd node.

FnC++ public headers

Function Documentation

FlashAttentionBwdResult oa::FnMatrix::flashAttentionCausalBwd(
	const Matrix & inQ,
	const Matrix & inK,
	const Matrix & inV,
	const Matrix & inOutput,
	const Matrix & inLogSumExp,
	const Matrix & inGradOutput,
	oa::F32 inScale
)

flashAttentionCausalBwd: explicit adjoint for the FlashAttention autograd node.

Parameters

inQ

const Matrix &

—

inK

const Matrix &

—

inV

const Matrix &

—

inOutput

const Matrix &

—

inLogSumExp

const Matrix &

—

inGradOutput

const Matrix &

—

inScale

oa::F32

—

Returns

FlashAttentionBwdResult

The declared return value.