oa::FnMatrix::swigluBwd

swigluBwd: backward pass for SwiGLU activation.

FnC++ public headers

Function Documentation

SwigluBwdResult oa::FnMatrix::swigluBwd(
	const Matrix & inGate,
	const Matrix & inUp,
	const Matrix & inOut,
	const Matrix & inGradOutput
)

swigluBwd: backward pass for SwiGLU activation.

inGate

const Matrix &

—

inUp

const Matrix &

—

inOut

const Matrix &

—

inGradOutput

const Matrix &

—

SwigluBwdResult

SwigluBwdResult with dGate, dUp