Fairness-Aware and Latency-Controllable Scheduling for Chunked-Prefill LLM Serving

Haoxin Liu, Jiayi Wang, Yueshen Xu +1

As large language models (LLMs) are increasingly deployed with highly heterogeneous workloads, chunked-prefill execution has emerged as a mainstream serving architecture. Balancing…