Throughput-Optimal Scheduling Algorithms for LLM Inference and AI Agents

J. G. Dai, Tianze Deng, Yueying Li +1

As demand for Large Language Models (LLMs) and AI agents grows rapidly, optimizing systems for efficient LLM inference becomes critical. While significant efforts have targeted sys…