Observation, Not Prediction: Conversation-Level Disaggregated Scheduling for Agentic Serving

Jianru Ding, Ryien Hosseini, Pouya Mahdi Gholami +2

LLM-based agents resolve a user task through many turns of dependent inference and tool calls, producing a workload whose total cost is unknown when the task arrives. Existing mult…