Dynamic Hierarchical Birkhoff-von Neumann Decomposition for All-to-All GPU Communication

Yen-Chieh Wu, Cheng-Shang Chang, Duan-Shin Lee +1

All-to-all GPU communication is a critical bottleneck in large-scale training clusters, where completion time is constrained by per-port bandwidth and can be severely impacted by t…