The Big Send-off: Scalable and Performant Collectives for Deep Learning

Siddharth Singh, Keshav Pradeep, Mahua Singh +2

Collective communication is becoming increasingly important in data center and supercomputer workloads with an increase in distributed AI related jobs. However, existing libraries…