4 papers · 1 filter
Share the Symbol, Not the Vector: Training-Free Fact Communication Across Language Models
Xin Li, Mengbing Liu, Hao Jiang +5
Multi-agent systems allow language models to solve tasks collaboratively by sharing complementary information. These systems typically communicate through text, while recent approa…
CoSA: Accelerating Long-Context Inference via Proxy-Kernel Co-Designed Sparse Attention
Yufei Xue, Lin Niu, Hong Liu +6
The quadratic cost of self-attention makes long-context inference prohibitively expensive, and proxy-based block-sparse attention has become a practical remedy. Existing methods ty…
Recall Before You Rank: Similarity-Guided Top- Reuse for Efficient Long-Context Attention
Wenshuai Yao, Wenyong Zhou, Hanyong Shao +5
Top- sparse attention reduces the cost of Softmax and value aggregation by attending to only a small subset of key--value (KV) entries. However, identifying this subset still re…
Sparse-BitNet: 1.58-bit LLMs are Naturally Friendly to Semi-Structured Sparsity
Di Zhang, Xun Wu, Shaohan Huang +9
Semi-structured N:M sparsity and low-bit quantization (e.g., 1.58-bit BitNet) are two promising approaches for improving the efficiency of large language models (LLMs), yet they ha…