7 papers
Refined Thompson Learning for Adaptive Bandits: Power-Efficient Flexibility Scheduling Across Data Centers
Zixi Chen, Yifu Ding, Ruicheng Ao +2
The rapid growth of large-scale AI workloads in data centers has placed increasing pressure on power grids in recent years. Since power systems must continuously balance supply and…
Online Linear Programming for Multi-Objective Routing in LLM Serving
Zixi Chen, Yinyu Ye, Zijie Zhou
We study the online routing problem in large language model serving, where requests arrive sequentially and must be dispatched to parallel decode workers under tight batch-size and…
Robust Restless Multi-Armed Bandit for Data Center Flexibility Services Through Virtual Machine Scheduling
Zixi Chen, Yifu Ding, Thomas Magnanti
Energy demands from data centers have surged and stressed the grid in recent years. Electric grids require balancing supply and demand every second, motivating demand response (red…
Tackling the Data-Parallel Load Balancing Bottleneck in LLM Serving: Practical Online Routing at Scale
Tianci Bu, Yuan Lyu, Zixi Chen +6
Data-parallel (DP) load balancing has emerged as a first-order bottleneck in large-scale LLM serving. When a model is sharded across devices via tensor parallelism (TP) or expert p…
Load Balancing with Evolving Workloads in Large-Scale Language Model Serving
Zixi Chen, Tianci Bu, Chendong Song +3
Serving large language models requires assigning requests whose workloads grow during processing and whose completion times are unknown. With a synchronization barrier, lightly loa…
Analytical Provisioning for Attention-FFN Disaggregated LLM Serving under Stochastic Workloads
Chendong Song, Meixuan Wang, Hang Zhou +5
Attentio-FFN disaggregation (AFD) is an emerging architecture for LLM decoding that separates state-heavy, KV-cache-dominated Attention computation from stateless, compute-intensiv…
Adaptively Robust LLM Inference Optimization under Prediction Uncertainty
Zixi Chen, Yinyu Ye, Zijie Zhou
We study the problem of optimizing Large Language Model (LLM) inference scheduling to minimize total latency. LLM inference is an online and multi-task service process and also hea…