TokenScale: Timely and Accurate Autoscaling for Disaggregated LLM Serving with Token Velocity

Ruiqi Lai, Hongrui Liu, Chengzhi Lu +6

The architectural shift to prefill/decode (PD) disaggregation in LLM serving improves resource utilization but struggles with the bursty nature of modern workloads. Existing autosc…