2 papers
cs.LG2025
SALS: Sparse Attention in Latent Space for KV cache Compression
Junlin Mu, Hantao Huang, Jihang Zhang +3
Large Language Models capable of handling extended contexts are in high demand, yet their inference remains challenging due to substantial Key-Value cache size and high memory band…
cs.LG2025
Online Pseudo-average Shifting Attention(PASA) for Robust Low-precision LLM Inference: Algorithms and Numerical Analysis
Long Cheng, Qichen Liao, Fan Wu +10
Attention calculation is extremely time-consuming for long-sequence inference tasks, such as text or image/video generation, in large models. To accelerate this process, we develop…