Taming Request Imbalance: SLO-Aware Scheduling for Disaggregated LLM Inference

Qipeng Wang

In production environments, large language model (LLM) serving is required to meet stringent service-level objectives (SLOs) amid highly variable request patterns. In practice, req…