1 paper
Hao Gu, Xintong Yang, Zhiwei Li +10
KV cache eviction must decide which entries carry information that the rest of the cache can no longer supply. A natural way to estimate this is a learnable gate: give each cached…