2 papers
cs.LG2026
Gandalf: Flash Shall Not Die! Increasing Flash Storage Lifetime for KV Cache Offloading in LLM Serving
Maharani A. P. Irawan, Kuntai Du, Jiayi Yao +4
KV caching reduces the serving cost of large language models (LLMs), but the growing KV cache footprint is increasingly exceeds GPU and host-memory capacity. Therefore, people use…
cs.AR2026
VeriCache: Turning Lossy KV Cache into Lossless LLM Inference
Jiayi Yao, Samuel Shen, Kuntai Du +7
The large size of the KV cache has become a major bottleneck for serving LLMs with increasing context lengths. In response, many KV cache compression methods, such as token droppin…