CADA: Communication-Adaptive Distributed Adam

Tianyi Chen, Ziye Guo, Yuejiao Sun +1

Stochastic gradient descent (SGD) has taken the stage as the primary workhorse for large-scale machine learning. It is often used with its adaptive variants such as AdaGrad, Adam,…