Random Attention: Rethinking KV Cache Eviction for Efficient Reasoning

Heng Wang, Jielin Qiu, Wenting Zhao +8

Large language models achieve superior performance on tasks that require extended reasoning, but long chains of thought make the KV cache a severe memory bottleneck. Existing KV ca…