SARIA: Persistence-Aware KV Caching with Coordinated DRAM--SSD Reload for LLM Agents

Hui Li, Qihui Zhou, Xiao Yan +7

Multi-turn LLM agents usually reuse long context prefixes, making key value (KV) caching crucial for reducing prefill latency. SSDs provide a low-cost extension to limited DRAM cap…