Adaptive Periodic Averaging: A Practical Approach to Reducing Communication in Distributed Learning

Peng Jiang, Gagan Agrawal

Stochastic Gradient Descent (SGD) is the key learning algorithm for many machine learning tasks. Because of its computational costs, there is a growing interest in accelerating SGD…