Adamas: Hadamard Sparse Attention for Efficient Long-Context Inference

Siyuan Yan, Guo-Qing Jiang, Yuchen Zhang +4

Large language models (LLMs) now support context windows of hundreds of thousands to millions of tokens, enabling applications such as long-document summarization, large-scale code…