5 papers
SpiderBoost and Momentum: Faster Stochastic Variance Reduction Algorithms
Zhe Wang, Kaiyi Ji, Yi Zhou +2
SARAH and SPIDER are two recently developed stochastic variance-reduced algorithms, and SPIDER has been shown to achieve a near-optimal first-order oracle complexity in smooth nonc…
Cubic Regularization with Momentum for Nonconvex Optimization
Zhe Wang, Yi Zhou, Yingbin Liang +1
Momentum is a popular technique to accelerate the convergence in practical training, and its impact on convergence guarantee has been well-studied for first-order algorithms. Howev…
A Note on Inexact Condition for Cubic Regularized Newton's Method
Zhe Wang, Yi Zhou, Yingbin Liang +1
This note considers the inexact cubic-regularized Newton's method (CR), which has been shown in \cite{Cartis2011a} to achieve the same order-level convergence rate to a secondary s…
Convergence of Cubic Regularization for Nonconvex Optimization under KL Property
Yi Zhou, Zhe Wang, Yingbin Liang
Cubic-regularized Newton's method (CR) is a popular algorithm that guarantees to produce a second-order stationary solution for solving nonconvex optimization problems. However, ex…
Stochastic Variance-Reduced Cubic Regularization for Nonconvex Optimization
Zhe Wang, Yi Zhou, Yingbin Liang +1
Cubic regularization (CR) is an optimization method with emerging popularity due to its capability to escape saddle points and converge to second-order stationary solutions for non…