Practical Online KV Cache Compaction for LLM Agents: An Empirical Study

Yujian Liu, Jiabao Ji, Li An +4

LLM agents accumulate long trajectories of reasoning steps, tool calls, and environment feedback, making the KV cache a major inference bottleneck. KV cache compaction can reduce t…