Showing 2026Show all
3 papers · 1 filter
cs.DC2026
Sample-Guided Exact Top-K Selection for Long-Context Sparse Attention
Siran Liu, Yang Xue, Theo Tang +11
Sparse attention bounds downstream attention work by retaining a fixed-size subset of indexed tokens, but its standalone exact Top- stage must still process materialized score r…
cs.CL2026
CoSA: Accelerating Long-Context Inference via Proxy-Kernel Co-Designed Sparse Attention
Yufei Xue, Lin Niu, Hong Liu +6
The quadratic cost of self-attention makes long-context inference prohibitively expensive, and proxy-based block-sparse attention has become a practical remedy. Existing methods ty…
cs.CL2026
RRAttention: Dynamic Block Sparse Attention via Per-Head Round-Robin Shifts for Long-Context Inference
Siran Liu, Guoxia Wang, Sa Wang +7
The quadratic complexity of attention mechanisms poses a critical bottleneck for large language models processing long contexts. While dynamic sparse attention methods offer input-…