An Efficient, Reliable and Observable Collective Communication Library in Large-scale GPU Training Clusters

Mingjun Zhang, Xiaohe Hu, Menghao Zhang +21

Large-scale LLM training requires collective communication libraries to exchange data among distributed GPUs. As a company dedicated to building and operating large-scale GPU train…