Showing cs.OSShow all
2 papers · 1 filter
cs.OS2026
Characterizing Predictability-Latency Trade-offs of KV-Cache SSD Offloading in LMCache for LLM Serving Systems
Ying He, Dingsen Shi, Yanbo Dai +5
KV-cache offload is widely used to stretch GPU memory for LLM serving, but its storage behavior has not been characterized at the block-device level. In this paper, we study LMCach…
cs.OS2026
Range: Decoupling KV-Cache Reuse Granularity from Storage Work
Dingsen Shi, Jiahao Zhang, Liqiang Lv +7
LLM serving reuses cached key-value (KV) state to skip prefill, and growing contexts push that state past GPU HBM and CPU DRAM onto NVMe SSDs, placing prefix restore on the critica…