Universal Stagewise Learning for Non-Convex Problems with Convergence on Averaged Solutions

Zaiyi Chen, Zhuoning Yuan, Jinfeng Yi +3

Although stochastic gradient descent (SGD) method and its variants (e.g., stochastic momentum methods, AdaGrad) are the choice of algorithms for solving non-convex problems (especi…