2 papers
cs.DC2026
Semantics, Workflows, and Infrastructure: Understanding Agent Serving at Production Scale
Yihao Zheng, Jingzhe Jiang, Dejiang Zhu +4
Large language model (LLM) agents execute applications through a workflow of inference requests with tool calls and user interactions. Serving these applications at production scal…
cs.DC2026
PackServe: SLO-Aware Request Scheduling for Agentic LLM Serving at Scale
Zhiyuan Tan, Dejiang Zhu, Jingzhe Jiang +4
Request scheduling is a key challenge in large-scale clusters serving agentic large language model (LLM) workloads. An effective scheduler must preserve key-value cache (KVC) reuse…