2 papers
cs.DC2025
Nexus:Proactive Intra-GPU Disaggregation of Prefill and Decode in LLM Serving
Xiaoxiang Shi, Colin Cai, Junjia Du +1
Monolithic serving with chunked prefill improves GPU utilization by batching prefill and decode together, but suffers from fine-grained phase interference. Engine-level prefill-dec…
cs.LG2025
Autellix: An Efficient Serving Engine for LLM Agents as General Programs
Michael Luo, Xiaoxiang Shi, Colin Cai +8
Large language model (LLM) applications are evolving beyond simple chatbots into dynamic, general-purpose agentic programs, which scale LLM calls and output tokens to help AI agent…