Optimal Scheduling Algorithms for LLM Inference: Theory and Practice

Agrim Bari, Parikshit Hegde, Gustavo de Veciana

With the growing use of Large Language Model (LLM)-based tools like ChatGPT, Perplexity, and Gemini across industries, there is a rising need for efficient LLM inference systems. T…