Block-Sparse Attention with Semantic-Geometric Decoupled Routing

Xinwei Long, Weigao Sun, Weibo Gao +6

Long-context inference has become a defining capability of large language models, but exact dense attention remains costly due to its quadratic scaling with sequence length. Block-…