306 citations · 764 across the 49 of their papers we have counts for
Showing 2018Show all
3 papers · 1 filter
cs.DC2018★ 4 cited
MG-WFBP: Efficient Data Communication for Distributed Synchronous SGD Algorithms
Shaohuai Shi, Xiaowen Chu, Bo Li
Distributed synchronous stochastic gradient descent has been widely used to train deep neural networks on computer clusters. With the increase of computational power, network commu…
cs.LG2018★ 306 cited
Highly Scalable Deep Learning Training System with Mixed-Precision: Training ImageNet in Four Minutes
Xianyan Jia, Shutao Song, Wei He +11
Synchronized stochastic gradient descent (SGD) optimizers with data parallelism are widely used in training large-scale deep neural networks. Although using larger mini-batch sizes…
cs.DC2018★ 8 cited
A DAG Model of Synchronous Stochastic Gradient Descent in Distributed Deep Learning
Shaohuai Shi, Qiang Wang, Xiaowen Chu +1
With huge amounts of training data, deep learning has made great breakthroughs in many artificial intelligence (AI) applications. However, such large-scale data sets present comput…