Automatic Configuration for Optimal Communication Scheduling in DNN Training

Yiqing Ma, Hao Wang, Yiming Zhang +1

ByteScheduler partitions and rearranges tensor transmissions to improve the communication efficiency of distributed Deep Neural Network (DNN) training. The configuration of hyper-p…