5 citations · 7 across the 4 of their papers we have counts for
4 papers · 1 filter
Ofan: Optimal Load Balancing for AI Training
Sarah McClure, Evyatar Cohen, Jakob Krebs +5
The extreme collective completion time (CCT) demands of AI workloads challenge existing packet spraying algorithms, which can have trouble efficiently load-balancing workloads that…
Incast-Free MoE Rate-Based Scheduling
Evyatar Cohen, Jose Yallouz, Alexander Shpiner +3
Mixture of Experts (MoE) architectures have become key to large language models; however, their typical round-robin (RR) scheduling introduces significant bottlenecks. In this pape…
Avoiding Cross-Datacenter Collective Congestion via Disaggregated Buffering
Mariano Scazzariello, Noga H. Rotman, Dima Gavrilenko +6
LLM training at the scale of tens of thousands of GPUs now spans multiple datacenters (DC), making cross-DC collectives over long-haul links unavoidable. A critical and overlooked…
Revisiting Network Support for RDMA
Radhika Mittal, Alexander Shpiner, Aurojit Panda +4
The advent of RoCE (RDMA over Converged Ethernet) has led to a significant increase in the use of RDMA in datacenter networks. To achieve good performance, RoCE requires a lossless…