Parallel and distributed asynchronous adaptive stochastic gradient methods

Yangyang Xu, Yibo Xu, Yonggui Yan +3

Stochastic gradient methods (SGMs) are the predominant approaches to train deep learning models. The adaptive versions (e.g., Adam and AMSGrad) have been extensively used in practi…