Revisiting Distributed Synchronous SGD

Xinghao Pan, Jianmin Chen, Rajat Monga +2

Distributed training of deep learning models on large-scale training data is typically conducted with asynchronous stochastic optimization to maximize the rate of updates, at the c…