36 citations · 36 across the 2 of their papers we have counts for
3 papers
cs.LG2021
Accelerate Distributed Stochastic Descent for Nonconvex Optimization with Momentum
Guojing Cong, Tianyi Liu
Momentum method has been used extensively in optimizers for deep learning. Recent studies show that distributed training through K-step averaging has many nice properties. We propo…
cs.LG2019★ 36 cited
Accelerating Data Loading in Deep Neural Network Training
Chih-Chieh Yang, Guojing Cong
Data loading can dominate deep neural network training time on large-scale systems. We present a comprehensive study on accelerating data loading performance in large-scale distrib…
cs.LG2019
A Distributed Hierarchical SGD Algorithm with Sparse Global Reduction
Fan Zhou, Guojing Cong
Reducing communication in training large-scale machine learning applications on distributed platform is still a big challenge. To address this issue, we propose a distributed hiera…