DeferKV: Rethinking Eviction Timing for One-Shot KV Cache Compression

Zhe Wang, Jiakai Li, Yujia Sun +2

Long-context large language models (LLMs) have demonstrated strong capabilities across a wide range of tasks, but the growing KV cache introduces substantial memory and inference o…