EvolKV: Evolutionary KV Cache Compression for LLM Inference

Bohan Yu, Yekun Chai

Existing key-value (KV) cache compression methods typically rely on heuristics, such as uniform cache allocation across layers or static eviction policies, however, they ignore the…