Efficient Agentic LLM Serving over SSD-based Sparse KV Storage

Wenhao He, Ping Zhang, Xiaohe Hu +5

Agentic sessions driven by Large language models (LLMs) often alternate between model inference and tool use, accumulating long histories across successive rounds. Serving these se…