3 papers
cs.DC2025
Load Balancing with Network Latencies via Distributed Gradient Descent
Santiago R. Balseiro, Vahab S. Mirrokni, Bartek Wydrowski
Motivated by the growing demand for serving large language model inference requests, we study distributed load balancing for global serving systems with network latencies. We consi…
cs.DC2024
Distributed Load Balancing with Workload-Dependent Service Rates
Wenxin Zhang, Santiago R. Balseiro, Robert Kleinberg +3
Modern service systems, including cloud platforms and large language model inference endpoints, must distribute jobs across servers whose processing speeds depend on current worklo…
cs.DC2023
Load is not what you should balance: Introducing Prequal
Bartek Wydrowski, Robert Kleinberg, Stephen M. Rumble +1
We present Prequal (Probing to Reduce Queuing and Latency), a load balancer for distributed multi-tenant systems. Prequal aims to minimize real-time request latency in the presence…