Don't Let a Few Network Failures Slow the Entire AllReduce

Peiqing Chen, Jiedong Jiang, Nengneng Yu +4

Network failures are among the most frequent hardware faults in large-scale GPU clusters and a leading cause of training-job interruptions. Modern collective communication librarie…