Exploiting Multicast for Accelerating Collective Communication

Chao Xu, Xu Zhang, Zihang Luo +5

Reducing collective communication latency is a critical goal for large model training and inference in both academia and industry. Many-to-many communications, such as AllGather an…