Efficient All-reduce for Distributed DNN Training in Optical Interconnect System

Fei Dai, Yawen Chen, Zhiyi Huang +2

Communication efficiency plays an important role in accelerating the distributed training of Deep Neural Networks (DNN). All-reduce is the crucial communication primitive to reduce…