RAP: KV-Cache Compression via RoPE-Aligned Pruning

Jihao Xin, Tian Lyu, David Keyes +2

Long-context inference in large language models (LLMs) is bottlenecked by the memory and compute of the key-value (KV) cache. Structured pruning is a direct way to shrink it: dropp…