3 papers
cs.DC2026
Semantics, Workflows, and Infrastructure: Understanding Agent Serving at Production Scale
Yihao Zheng, Jingzhe Jiang, Dejiang Zhu +4
Large language model (LLM) agents execute applications through a workflow of inference requests with tool calls and user interactions. Serving these applications at production scal…
cs.DC2026
PackServe: SLO-Aware Request Scheduling for Agentic LLM Serving at Scale
Zhiyuan Tan, Dejiang Zhu, Jingzhe Jiang +4
Request scheduling is a key challenge in large-scale clusters serving agentic large language model (LLM) workloads. An effective scheduler must preserve key-value cache (KVC) reuse…
cs.DC2025
Janus: Disaggregating Attention and Experts for Scalable MoE Inference
Zhexiang Zhang, Ye Wang, Yumiao Zhao +9
Serving large Mixture-of-Experts (MoE) models is challenging because of their large memory footprints, heterogeneous resource demands, and highly dynamic inference workloads. Most…