Back from the Future: Key-Value Cache Management by Counter-Causal Surprise

Stephen Gould, Anton van den Hengel

Key-value (KV) cache management through compression and eviction strategies has emerged as an important research direction in recent years. Computational demands of large language…