3 papers
cs.CL2025
Adamas: Hadamard Sparse Attention for Efficient Long-Context Inference
Siyuan Yan, Guo-Qing Jiang, Yuchen Zhang +4
Large language models (LLMs) now support context windows of hundreds of thousands to millions of tokens, enabling applications such as long-document summarization, large-scale code…
cs.CL2025
dots.llm1 Technical Report
Bi Huo, Bin Tu, Cheng Qin +24
Mixture of Experts (MoE) models have emerged as a promising paradigm for scaling language models efficiently by activating only a subset of parameters for each input token. In this…
cs.CL2025
Scaling Laws for Speculative Decoding
Siyuan Yan, Mo Zhu, Guo-qing Jiang +8
The escalating demand for efficient decoding in large language models (LLMs) is particularly critical for reasoning-intensive architectures like OpenAI-o3 and DeepSeek-R1, which de…