Ethereal: Divide and Conquer Network Load Balancing in Large-Scale Distributed Training

Vamsi Addanki, Prateesh Goyal, Ilias Marinos +1

Large-scale distributed training in production datacenters constitutes a challenging workload bottlenecked by network communication. In response, both major industry players (e.g.,…