ResidualKV: Residual-Based KV Cache Compression for Efficient Long-Context Inference

Jitai Hao, Qiang Huang, Yaowei Wang +2

Efficient long-context inference faces two coupled bottlenecks: KV-cache memory grows linearly with context length, while attention computation grows quadratically. Existing approa…