Leader Stochastic Gradient Descent for Distributed Training of Deep Learning Models: Extension

Yunfei Teng, Wenbo Gao, Francois Chalus +3

We consider distributed optimization under communication constraints for training deep learning models. We propose a new algorithm, whose parameter updates rely on two forces: a re…