2 papers
cs.DC2026
Discovering KV Cache Eviction Policies via LLM-Guided Program Evolution
Pratik Poudel, Yanzhao Wu, Sumit Jha +1
KV cache compression is critical for long-context inference, yet effective eviction policies remain difficult to design: existing prefill-stage methods often rely on hand-crafted s…
cs.LG2025
Stateful KV Cache Management for LLMs: Balancing Space, Time, Accuracy, and Positional Fidelity
Pratik Poudel
The Key-Value (KV) cache is integral to efficient autoregressive inference in large language models (LLMs), yet its unbounded growth in stateful multi-turn scenarios presents major…