Range: Decoupling KV-Cache Reuse Granularity from Storage Work

Dingsen Shi, Jiahao Zhang, Liqiang Lv +7

LLM serving reuses cached key-value (KV) state to skip prefill, and growing contexts push that state past GPU HBM and CPU DRAM onto NVMe SSDs, placing prefix restore on the critica…