oa::Rope

Rotary position embedding over a flattened [tokens, heads * head_dim] input.

ClC++ public headers

Inheritance

public Module

oa::Matrix oa::Rope::forward(const oa::Matrix & inInput)

oa::Rope::Rope(
	oa::I32 inNumHeads,
	oa::I32 inHeadDim,
	oa::F32 inThetaBase = 10000.0F
)

No public source comment is attached to this declaration.

inNumHeads

oa::I32

—

inHeadDim

oa::I32

—

inThetaBase

oa::F32

Default: 10000.0F

oa::Matrix oa::Rope::forward(
	const oa::Matrix & inInput
)

No public source comment is attached to this declaration.

inInput

const oa::Matrix &

—

oa::Matrix

The declared return value.