4 papers
CoSA: Accelerating Long-Context Inference via Proxy-Kernel Co-Designed Sparse Attention
Yufei Xue, Lin Niu, Hong Liu +6
The quadratic cost of self-attention makes long-context inference prohibitively expensive, and proxy-based block-sparse attention has become a practical remedy. Existing methods ty…
Recall Before You Rank: Similarity-Guided Top- Reuse for Efficient Long-Context Attention
Wenshuai Yao, Wenyong Zhou, Hanyong Shao +5
Top- sparse attention reduces the cost of Softmax and value aggregation by attending to only a small subset of key--value (KV) entries. However, identifying this subset still re…
SlideSparse: Fast and Flexible (2N-2):2N Structured Sparsity
Hanyong Shao, Yingbo Hao, Ting Song +10
NVIDIA's 2:4 Sparse Tensor Cores deliver 2x throughput but demand strict 50% pruning -- a ratio that collapses LLM reasoning accuracy (Qwen3: 54% to 15%). Milder patter…
Sparse-BitNet: 1.58-bit LLMs are Naturally Friendly to Semi-Structured Sparsity
Di Zhang, Xun Wu, Shaohan Huang +9
Semi-structured N:M sparsity and low-bit quantization (e.g., 1.58-bit BitNet) are two promising approaches for improving the efficiency of large language models (LLMs), yet they ha…