oa::FnMatrix::swiglu

swiglu: SwiGLU activation: out = SiLU(gate) * up.

FnC++ public headers

Function Documentation

oa::Matrix oa::FnMatrix::swiglu(
	const oa::Matrix & inA,
	const oa::Matrix & inB
)

swiglu: SwiGLU activation: out = SiLU(gate) * up.

inA

const oa::Matrix &

—

inB

const oa::Matrix &

—

oa::Matrix

The declared return value.