Preserving Provenance in Shared KV Caches for LLM Serving

Wei Song, Yuxin Cao, Xi Zheng +2

Production LLM serving stacks combine an inference engine's local prefix cache with a shared KV-cache tier for fleet-wide reuse. The local cache distinguishes requests by adapter,…