From Tensor Buffer to Distributed Memory Hierarchy: A Survey of KV Cache Management for LLM Serving

Jie Li, Tongyang Wang, Yong Chen

The key-value (KV) cache has become a first-order memory object in LLM serving rather than a temporary per-request tensor. This survey classifies more than thirty KV-management sys…