FlatCake: A Flat KVCache Storage Architecture for LLM Serving

Xin Huang, Teng Ma, Zhuoran Li +7

KV cache reuse accelerates LLM serving, but existing systems manage host DRAM and NVMe SSD as a tiered hierarchy where DRAM acts as an intermediate cache. Production LLM workloads…