#counter-causal attention

Back from the Future: Key-Value Cache Management by Counter-Causal Surprise

Stephen Gould, Anton van den Hengel

The paper introduces a KV cache eviction method for large language models that removes entries predicted well by future tokens using a counter‑causal attention pass, improving memo…