4 papers
Agentic Coding in the Wild: Characterizing GitHub Copilot Traces at Production Scale
Banruo Liu, Haoran Qiu, Ãñigo Goiri +3
AI coding agents like GitHub Copilot, Claude Code, and Codex interleave multi-step LLM inference with tool execution, creating a workload different from chatbots. We present the fi…
Compass: SLO-aware Query Planner for Compound AI Serving at Scale
Banruo Liu, Wei-Yu Lin, Minghao Fang +2
The rise of compound AI serving that integrates multiple operators in a pipeline enables end-user applications such as generative AI-powered meeting companions, autonomous driving,…
JITServe: SLO-aware LLM Serving with Imprecise Request Information
Wei Zhang, Zhiyu Wu, Yi Mu +5
The integration of Large Language Models (LLMs) into applications ranging from interactive chatbots to multi-agent systems has introduced a wide spectrum of service-level objective…
Single-agent or Multi-agent Systems? Why Not Both?
Mingyan Gao, Yanzi Li, Banruo Liu +4
Multi-agent systems (MAS) decompose complex tasks and delegate subtasks to different large language model (LLM) agents and tools. Prior studies have reported the superior accuracy…