oa::ByteEmbedding
BYTE EMBEDDING - The 256-entry lookup table Maps each byte value (0-255) to a d_model dimensional vector. input: [batch, seq_len] of UInt8 output: [batch, seq_len, d_model] of Float32 This replaces the massive 30k-100k token embedding tables in GPT/LLaMA. Ours is always 256 x d_model. Tiny. Fast. Universal.
Inheritance
public Module
Public Methods
Constructor & Destructor Documentation
Public Method Documentation
No public source comment is attached to this declaration.
Returns
oa::I32The declared return value.
[batch, seq] -> [batch, seq, d_model]
Parameters
inByteIdsconst Matrix &—
Returns
MatrixThe declared return value.