CacheFlow: Efficient LLM Serving via Automated 3D-Parallel KV Cache Restoration

Sean Nian, Jiahao Fang, Hanzhang Shen +3

KV cache restoration is becoming a major bottleneck in long-context LLM serving, including multi-turn conversations, agentic workloads, and RAG. Existing advances largely treat res…