2 citations · 2 across the 3 of their papers we have counts for
Showing cs.DCShow all
2 papers · 1 filter
cs.DC2026
DLB: Distributed Load Balancing at Scale for Generative AI Inference
Santiago R. Balseiro, Bartek Wydrowski, Sameer Agarwal +11
The reliance on scarce and expensive accelerators such as GPUs and TPUs in modern datacenters places unprecedented demands on backend infrastructure. For workloads characterized by…
cs.DC2023★ 2 cited
Load is not what you should balance: Introducing Prequal
Bartek Wydrowski, Robert Kleinberg, Stephen M. Rumble +1
We present Prequal (Probing to Reduce Queuing and Latency), a load balancer for distributed multi-tenant systems. Prequal aims to minimize real-time request latency in the presence…