Risk-Controlled KV-Cache Eviction: From Memory Budgets to Risk Targets

Beomgu Kang, SoJin Yun, Hojoon Kim +1

KV-cache eviction is typically evaluated through average quality-memory trade-offs, yet a small average loss can hide requests whose utility degrades materially. We reformulate evi…