On the Convergence of Decentralized Adaptive Gradient Methods

Xiangyi Chen, Belhal Karimi, Weijie Zhao +1

Adaptive gradient methods including Adam, AdaGrad, and their variants have been very successful for training deep learning models, such as neural networks. Meanwhile, given the nee…