3 papers
cs.DC2026
Semantics, Workflows, and Infrastructure: Understanding Agent Serving at Production Scale
Yihao Zheng, Jingzhe Jiang, Dejiang Zhu +4
Large language model (LLM) agents execute applications through a workflow of inference requests with tool calls and user interactions. Serving these applications at production scal…
cs.DC2026
PackServe: SLO-Aware Request Scheduling for Agentic LLM Serving at Scale
Zhiyuan Tan, Dejiang Zhu, Jingzhe Jiang +4
Request scheduling is a key challenge in large-scale clusters serving agentic large language model (LLM) workloads. An effective scheduler must preserve key-value cache (KVC) reuse…
cs.DC2026
TENT: A Declarative Slice Spraying Engine for Performant and Resilient Data Movement in Disaggregated LLM Serving
Feng Ren, Ruoyu Qin, Teng Ma +16
Modern GPU clusters rely on complex, heterogeneous interconnects. As large language model (LLM) serving shifts toward agentic reasoning, KVCache becomes a first-class mobile asset,…