most citedTowards Efficient Large Language Model Serving: A Survey on System-Aware KV Cache Optimization

2 citations · 2 across the 3 of their papers we have counts for

collaborators
Showing cs.LGShow all

2 papers · 1 filter

cs.LG2026

Does Execution Require Target KV Fidelity? A Mixed-Fidelity KV Runtime for LLM Serving

Jiantong Jiang, Yue Yang, Peiyu Yang +1

Large language model (LLM) serving is increasingly constrained by the GPU memory consumed by key-value (KV) caches. Existing compression, eviction, and offloading techniques allevi…

cs.LG2026★ 2 cited

Towards Efficient Large Language Model Serving: A Survey on System-Aware KV Cache Optimization

Jiantong Jiang, Peiyu Yang, Rui Zhang +1

Despite the rapid advancements of large language models (LLMs), LLM serving systems remain memory-intensive and costly. The key-value (KV) cache, which stores KV tensors during aut…