#kv cache management
topickv cache management
2 papers · 1 filter
cs.LG2026
Back from the Future: Key-Value Cache Management by Counter-Causal Surprise
Stephen Gould, Anton van den Hengel
The paper introduces a KV cache eviction method for large language models that removes entries predicted well by future tokens using a counter‑causal attention pass, improving memo…
cs.LG2026
FastTPS: An Optimized Method for LLM Token Phase for AI accelerators
Wenzong Yang, Danyang Zhang, Kun Cao +17
FastTPS is a set of techniques that reduce memory traffic and fuse operations during the token‑phase of large language model inference on AI accelerators, achieving up to a 6× spee…