2 papers
cs.DC2026
From Tensor Buffer to Distributed Memory Hierarchy: A Survey of KV Cache Management for LLM Serving
Jie Li, Tongyang Wang, Yong Chen
The key-value (KV) cache has become a first-order memory object in LLM serving rather than a temporary per-request tensor. This survey classifies more than thirty KV-management sys…
cs.LG2025
TokenPowerBench: Benchmarking the Power Consumption of LLM Inference
Chenxu Niu, Wei Zhang, Jie Li +4
Large language model (LLM) services now answer billions of queries per day, and industry reports show that inference, not training, accounts for more than 90% of total power consum…