Communication optimization strategies for distributed deep neural network training: A survey

Shuo Ouyang, Dezun Dong, Yemao Xu +1

Recent trends in high-performance computing and deep learning have led to the proliferation of studies on large-scale deep neural network training. However, the frequent communicat…