2 citations · 2 across the 4 of their papers we have counts for
4 papers
DLB: Distributed Load Balancing at Scale for Generative AI Inference
Santiago R. Balseiro, Bartek Wydrowski, Sameer Agarwal +11
The reliance on scarce and expensive accelerators such as GPUs and TPUs in modern datacenters places unprecedented demands on backend infrastructure. For workloads characterized by…
Load Balancing with Network Latencies via Distributed Gradient Descent
Santiago R. Balseiro, Vahab S. Mirrokni, Bartek Wydrowski
Motivated by the growing demand for serving large language model inference requests, we study distributed load balancing for global serving systems with network latencies. We consi…
Distributed Load Balancing with Workload-Dependent Service Rates
Wenxin Zhang, Santiago R. Balseiro, Robert Kleinberg +3
Modern service systems, including cloud platforms and large language model inference endpoints, must distribute jobs across servers whose processing speeds depend on current worklo…
Load is not what you should balance: Introducing Prequal
Bartek Wydrowski, Robert Kleinberg, Stephen M. Rumble +1
We present Prequal (Probing to Reduce Queuing and Latency), a load balancer for distributed multi-tenant systems. Prequal aims to minimize real-time request latency in the presence…