Double Quantization for Communication-Efficient Distributed Optimization

Yue Yu, Jiaxiang Wu, Longbo Huang

Modern distributed training of machine learning models suffers from high communication overhead for synchronizing stochastic gradients and model parameters. In this paper, to reduc…