Efficiency and Cost Alignment in Batched LLM Serving via Resource-Fair Scheduling

Dayi Yao, Zijie Zhou

This paper studies a resource-allocation inefficiency in batched large language model (LLM) serving: heterogeneous requests that share a decode batch impose max-driven computationa…