PRISM: Fast Online LLM Serving via Scheduling-Memory Co-design

Xingyu Qu, Tianhao Lin, Yiqi Li +2

Modern online large language model (LLM) services, such as Retrieval-Augmented Generation (RAG) and agent systems, increasingly expose two prominent characteristics: prompt segment…