Thought-Aware KV Cache Compaction for Reasoning via Adaptive Attention Matching

Yang Liu, Bin Chong, Chongyang Zhang +3

Reasoning language models generate lengthy chain-of-thought (CoT) sequences whose key-value (KV) cache grows linearly and becomes a memory bottleneck during decoding. Existing comp…