EfficientAgent: What Makes KV Cache Offloading Work for Concurrent Agents?

Kunming Shao, Jierun Chen, Jiangnan Yu +7

LLM agents resend their whole conversation on every turn, and most of it was already processed on the previous turn. Serving systems avoid recomputing it by caching its key-value (…