Unified Convergence Analysis of Stochastic Momentum Methods for Convex and Non-convex Optimization

Tianbao Yang, Qihang Lin, Zhe Li

Recently, {\it stochastic momentum} methods have been widely adopted in training deep neural networks. However, their convergence analysis is still underexplored at the moment, in…