AgentKV: Phase-Aware KV Eviction for Agentic LLMs

Taowen Tony Liu, Jeffrey T. H. Wong, Can Xiao +3

Agentic serving can consume orders of magnitude more tokens than chatbot workloads, stressing both KV-cache capacity and decode-time bandwidth. Most KV-eviction methods score cache…