PackServe: SLO-Aware Request Scheduling for Agentic LLM Serving at Scale

Zhiyuan Tan, Dejiang Zhu, Jingzhe Jiang +4

Request scheduling is a key challenge in large-scale clusters serving agentic large language model (LLM) workloads. An effective scheduler must preserve key-value cache (KVC) reuse…