7 papers
LoGo: Token-Level Dynamic Local-Global Attention
Yuqi Pan, Zheng Li, Bohao Tang +2
As context lengths scale, attention increasingly becomes a primary computational bottleneck in large language models. Standard Transformers remain powerful but computationally inef…
Modular TTT: Rethinking Test-Time Training as Composable Modules
Bohao Tang, Zhen Qin, Yuqi Pan +3
Test-time training (TTT) views sequence modeling as an online learning problem in which fast weights are updated by an internal learning rule. Despite the growing number of TTT var…
DFlare: Scaling Up Draft Capacity for Block Diffusion Speculative Decoding
Jiebin Zhang, Zhenghan Yu, Song Liu +9
Block diffusion speculative decoding accelerates LLM inference by predicting all tokens within a block simultaneously for the target model to verify in parallel. Predicting an enti…
Learning to Draft: Adaptive Speculative Decoding with Reinforcement Learning
Jiebin Zhang, Zhenghan Yu, Liang Wang +8
Speculative decoding accelerates large language model (LLM) inference by using a small draft model to generate candidate tokens for a larger target model to verify. The efficacy of…
Seed Diffusion: A Large-Scale Diffusion Language Model with High-Speed Inference
Yuxuan Song, Zheng Zhang, Cheng Luo +19
We present Seed Diffusion Preview, a large-scale language model based on discrete-state diffusion, offering remarkably fast inference speed. Thanks to non-sequential, parallel gene…
Scaling Linear Attention with Sparse State Expansion
Yuqi Pan, Yongqi An, Zheng Li +6
The Transformer architecture, despite its widespread success, struggles with long-context scenarios due to quadratic computation and linear memory growth. While various linear atte…
A Systematic Analysis of Hybrid Linear Attention
Dustin Wang, Rui-Jie Zhu, Steven Abreu +9
Transformers face quadratic complexity and memory issues with long sequences, prompting the adoption of linear attention mechanisms using fixed-size hidden states. However, linear…