Behavior-Preserving KV Cache Compression

Doo Hwan Hwang, Junyoung Jang, Junho Na +2

KV caches are a major bottleneck in long-context inference and long-form generation with large language models. Existing training-free eviction policies largely rely on proxy impor…