Progressive Sparse Attention: Algorithm and System Co-design for Efficient Attention in LLM Serving

Qihui Zhou, Peiqi Yin, Pengfei Zuo +1

Processing long contexts has become a critical capability for modern large language models (LLMs). However, serving long-context LLMs comes with significant inference costs due to…