Rethinking KV Cache Eviction via a Unified Information-Theoretic Objective

Jiaming Yang, Chenwei Tang, Liangli Zhen +1

Key-Value (KV) caching is essential for large language model inference, yet its memory overhead poses a critical bottleneck for long-context generation. Existing eviction policies…