4 papers
Block Sparse Attention with Log-Linear Complexity
Bohao Tang, Zhen Qin, Yuqi Pan +2
Scaling language models to long contexts is limited by the quadratic cost of self-attention. Block sparse attention offers an efficient alternative, but selecting the retained bloc…
LoGo: Token-Level Dynamic Local-Global Attention
Yuqi Pan, Zheng Li, Bohao Tang +2
As context lengths scale, attention increasingly becomes a primary computational bottleneck in large language models. Standard Transformers remain powerful but computationally inef…
TsuGO: Probing Search Efficiency in LLM Reasoning via Go Life-and-Death Problems
Shunwen Bai, Ziping Ma, Chaoyang Zhang +4
The evaluation of LLM reasoning is moving from final-answer accuracy to process-level assessment, yet existing methods still fail to capture how models plan reasoning paths and all…
Modular TTT: Rethinking Test-Time Training as Composable Modules
Bohao Tang, Zhen Qin, Yuqi Pan +3
Test-time training (TTT) views sequence modeling as an online learning problem in which fast weights are updated by an internal learning rule. Despite the growing number of TTT var…