AnchorKV: Safety-Aware KV Cache Compression via Soft Penalty with a Refusal Anchor

Ning Ni, Yingjie Lao

Large language models (LLMs) outperform earlier architectures on generative inference and long-context tasks, but their large size introduces significant challenges in memory usage…