S2 Reducer: High-Performance Sparse Communication to Accelerate Distributed Deep Learning

Keshi Ge, Yongquan Fu, Zhiquan Lai +2

Distributed stochastic gradient descent (SGD) approach has been widely used in large-scale deep learning, and the gradient collective method is vital to ensure the training scalabi…