KVzap: Fast, Adaptive, and Faithful KV Cache Pruning

Simon Jegou, Maximilian Jeblick

Growing context lengths in transformer-based language models have made the key-value (KV) cache a critical inference bottleneck. While many KV cache pruning methods have been propo…