A Distributed SGD Algorithm with Global Sketching for Deep Learning Training Acceleration

LingFei Dai, Boyu Diao, Chao Li +1

Distributed training is an effective way to accelerate the training process of large-scale deep learning models. However, the parameter exchange and synchronization of distributed…