7 papers
Refined Thompson Learning for Adaptive Bandits: Sustainable Power-Efficient Flexibility Scheduling Across Data Centers
Yifu Ding, Zixi Chen, Ruicheng Ao +2
The rapid rise in energy consumption from large-scale AI workloads in data centers placed the increasing pressures on power grids in recent years. Since grids must maintain real-ti…
Online Linear Programming for Multi-Objective Routing in LLM Serving
Zixi Chen, Yinyu Ye, Zijie Zhou
We study the online routing problem in large language model serving, where requests arrive sequentially and must be dispatched to parallel decode workers under tight batch-size and…
Robust Restless Multi-Armed Bandit for Data Center Flexibility Services Through Virtual Machine Scheduling
Zixi Chen, Yifu Ding, Thomas Magnanti
Energy demands from data centers have surged and stressed the grid in recent years. Electric grids require balancing supply and demand every second, motivating demand response (red…
Analytical Provisioning for Attention-FFN Disaggregated LLM Serving under Stochastic Workloads
Chendong Song, Meixuan Wang, Hang Zhou +5
Attentio-FFN disaggregation (AFD) is an emerging architecture for LLM decoding that separates state-heavy, KV-cache-dominated Attention computation from stateless, compute-intensiv…
Tackling the Data-Parallel Load Balancing Bottleneck in LLM Serving: Practical Online Routing at Scale
Tianci Bu, Yuan Lyu, Zixi Chen +6
Data-parallel (DP) load balancing has emerged as a first-order bottleneck in large-scale LLM serving. When a model is sharded across devices via tensor parallelism (TP) or expert p…
A Universal Load Balancing Principle and Its Application to Large Language Model Serving
Zixi Chen, Tianci Bu, Chendong Song +3
Over 40% of computational power in Large Language Model (LLM) serving systems can be systematically wasted - not from hardware limits, but from load imbalance in barrier-synchroniz…