3 papers
cs.LG2026
Reversible Foundations: Training a 120B Sparse MoE through State-Preserving Scaling
Rohan Shravan
This paper reports on training a hundred-billion-parameter sparse mixture of experts on a single eight-GPU node, end to end. LightningLM 0.1V is a recurrence-backbone language mode…
cs.CL2026
Kronecker Embeddings: Byte-Level Structured Token Representations for Parameter-Efficient Language Models
Rohan Shravan
Large language models route every input through a learned embedding table of shape |V| x d_model, consuming hundreds of millions to billions of trainable parameters at frontier sca…
cs.CL2026
BrahmicTokenizer-131K: An Indic-Capable Drop-In Replacement for o200k_base
Rohan Shravan
We present BrahmicTokenizer-131K, a 131,072-vocabulary byte-level BPE tokenizer that closes the Brahmic compression gap at the 131K-vocabulary class while preserving the English, E…