MemDecay: Region-Aware KV Cache Eviction for Efficient LLM Agent Inference

Venkatesha Matam, Keon Kim

Large language model (LLM) agents accumulate heterogeneous context, including system instructions, plans, user turns, retrieved documents, tool outputs, and intermediate reasoning,…