HiSparse: Scaling Sparse-Attention Decoding with Hierarchical KV Cache Management

Zhiqiang Xie, Zhangheng Huang, Tingwei Huang +3

Top-k sparse attention makes long-context LLM decoding cheap to compute: each step reads only a few thousand selected KV entries rather than the full context. Serving systems, howe…