activity
20232026
most citedLMCache: An Efficient KV Cache Layer for Enterprise-Scale LLM Inference

2 citations · 2 across the 9 of their papers we have counts for

collaborators
Showing cs.OSShow all

2 papers · 1 filter

cs.OS2025

EVICPRESS: Joint KV-Cache Compression and Eviction for Efficient LLM Serving

Shaoting Feng, Yuhan Liu, Hanchen Li +11

Reusing KV cache is essential for high efficiency of Large Language Model (LLM) inference systems. With more LLM users, the KV cache footprint can easily exceed GPU memory capacity…

cs.OS2025

AdaptCache: KV Cache Native Storage Hierarchy for Low-Delay and High-Quality Language Model Serving

Shaoting Feng, Hanchen Li, Kuntai Du +8

Large language model (LLM) applications often reuse previously processed context, such as chat history and documents, which introduces significant redundant computation. Existing L…