Benchmarking KV-Cache Optimizations across Task Quality and System Performance for Long-Context Serving

Nikita Agrawal, Ruben Mayer

Large language model serving is increasingly limited by KV-cache growth under long-context workloads, yet existing KV-cache compression techniques are difficult to compare because…