Akashic: A Low-Overhead LLM Inference Service with MemAttention

Yang Liu, Zhaokai Luo, Huayi Jin +7

Recent LLM-based agent systems continuously accumulate context across multi-turn interactions, tool invocations, and cross-session workflows. Replaying the full history for every r…