48 citations · 52 across the 4 of their papers we have counts for
Showing 2019Show all
2 papers · 1 filter
cs.DC2019
Auto-Precision Scaling for Distributed Deep Learning
Ruobing Han, James Demmel, Yang You
It has been reported that the communication cost for synchronizing gradients can be a bottleneck, which limits the scalability of distributed deep learning. Using low-precision gra…
cs.DC2019★ 48 cited
Optimizing Network Performance for Distributed DNN Training on GPU Clusters: ImageNet/AlexNet Training in 1.5 Minutes
Peng Sun, Wansen Feng, Ruobing Han +2
It is important to scale out deep neural network (DNN) training for reducing model training time. The high communication overhead is one of the major performance bottlenecks for di…