oa::Swiglu

Swiglu: SwiGLU MLP layer with optional weight concatenation.

Inheritance

public Module

Public Methods

Matrix oa::Swiglu::forward(const Matrix & inInput)

Constructor & Destructor Documentation

oa::Swiglu::Swiglu( oa::I32 inFeatures, oa::I32 inIntermediateSize, bool inBias = false )
No public source comment is attached to this declaration.

Parameters

inFeatures
oa::I32

inIntermediateSize
oa::I32

inBias
bool

Default: false

Public Method Documentation

Matrix oa::Swiglu::forward( const Matrix & inInput )
No public source comment is attached to this declaration.

Parameters

inInput
const Matrix &

Returns

Matrix

The declared return value.