Revisiting Distributed Synchronous SGD

Jianmin Chen, Xinghao Pan, Rajat Monga +2

Distributed training of deep learning models on large-scale training data is typically conducted with asynchronous stochastic optimization to maximize the rate of updates, at the c…