Autellix: An Efficient Serving Engine for LLM Agents as General Programs

Michael Luo, Xiaoxiang Shi, Colin Cai +8

Large language model (LLM) applications are evolving beyond simple chatbots into dynamic, general-purpose agentic programs, which scale LLM calls and output tokens to help AI agent…