7 papers
MRNN: Non-Linear RNNs with Matrix-Valued States for Scalable Language Modeling
Mayank Mishra, Shawn Tan, Ion Stoica +2
Transformers are highly parallel but are limited to computations in the TC complexity class, excluding tasks such as entity tracking and code execution that provably require gr…
SonicMoE: Accelerating MoE with IO and Tile-aware Optimizations
Wentao Guo, Mayank Mishra, Xinle Cheng +2
Mixture of Experts (MoE) models have emerged as the de facto architecture for scaling up language models without significantly increasing the computational cost. Recent MoE models…
Mamba-3: Improved Sequence Modeling using State Space Principles
Aakash Lahoti, Kevin Y. Li, Berlin Chen +5
Scaling inference-time compute has emerged as an important driver of LLM performance, making inference efficiency a central focus of model design alongside model quality. While the…
Log-Linear Attention
Han Guo, Songlin Yang, Tarushii Goel +3
The attention mechanism in Transformers is an important primitive for accurate and scalable sequence modeling. Its quadratic-compute and linear-memory complexity however remain sig…
Hardware-Efficient Attention for Fast Decoding
Ted Zadouri, Hubert Strauss, Tri Dao
LLM decoding is bottlenecked for large batches and long contexts by loading the key-value (KV) cache from high-bandwidth memory, which inflates per-token latency, while the sequent…
Long-Context State-Space Video World Models
Ryan Po, Yotam Nitzan, Richard Zhang +5
Video diffusion models have recently shown promise for world modeling through autoregressive frame prediction conditioned on actions. However, they struggle to maintain long-term m…