oa::VramBudgetConfig

VRAM BUDGET — auto-tune batch_size * seq_len from free VRAM (implementation in budget.cpp)

ClC++ public headers

Public Data Members

bytesPerParamoa::I32field

maxBatchSizeoa::I32field

maxSeqLenoa::I32field

minBatchSizeoa::I32field

minSeqLenoa::I32field

optimizerStatesPerParamoa::I32field

preferredBatchSizeoa::I32field

preferredSeqLenoa::I32field

safetyMarginPercentoa::F32field