Governing the KV Cache: Preventing Timing Side-Channel Leakage in Multi-Tenant LLM Inference

Tejasvi C. Addagada

The key-value (KV) cache is the primary throughput optimization in modern large language model (LLM) inference, enabling prefix reuse across requests. In multi-tenant deployments t…