Efficient Multi-round LLM Inference over Disaggregated Serving

Wenhao He, Youhe Jiang, Penghao Zhao +4

With the rapid evolution of Large Language Models (LLMs), multi-round workflows, such as autonomous agents and iterative retrieval, have become increasingly prevalent. However, thi…