52 citations · 289 across the 30 of their papers we have counts for
11 papers · 1 filter
Unified Convergence Analysis for Adaptive Optimization with Moving Average Estimator
Zhishuai Guo, Yi Xu, Wotao Yin +2
Although adaptive optimization algorithms have been successful in many applications, there are still some mysteries in terms of convergence analysis that have not been unraveled. T…
Optimal Epoch Stochastic Gradient Descent Ascent Methods for Min-Max Optimization
Yan Yan, Yi Xu, Qihang Lin +2
Epoch gradient descent method (a.k.a. Epoch-GD) proposed by Hazan and Kale (2011) was deemed a breakthrough for stochastic strongly convex minimization, which achieves the optimal…
On the Convergence of (Stochastic) Gradient Descent with Extrapolation for Non-Convex Optimization
Yi Xu, Zhuoning Yuan, Sen Yang +2
Extrapolation is a well-known technique for solving convex optimization and variational inequalities and recently attracts some attention for non-convex optimization. Several recen…
Non-asymptotic Analysis of Stochastic Methods for Non-Smooth Non-Convex Regularized Problems
Yi Xu, Rong Jin, Tianbao Yang
Stochastic Proximal Gradient (SPG) methods have been widely used for solving optimization problems with a simple (possibly non-smooth) regularizer in machine learning and statistic…
Stochastic Optimization for DC Functions and Non-smooth Non-convex Regularizers with Non-asymptotic Convergence
Yi Xu, Qi Qi, Qihang Lin +2
Difference of convex (DC) functions cover a broad family of non-convex and possibly non-smooth and non-differentiable functions, and have wide applications in machine learning and…
Frank-Wolfe Method is Automatically Adaptive to Error Bound Condition
Yi Xu, Tianbao Yang
Error bound condition has recently gained revived interest in optimization. It has been leveraged to derive faster convergence for many popular algorithms, including subgradient me…
Katalyst: Boosting Convex Katayusha for Non-Convex Problems with a Large Condition Number
Zaiyi Chen, Yi Xu, Haoyuan Hu +1
In this paper, we propose a new SVRG-style acceleated stochastic algorithm for solving a family of non-convex optimization problems whose objective consists of a sum of smooth…
NEON+: Accelerated Gradient Methods for Extracting Negative Curvature for Non-Convex Optimization
Yi Xu, Rong Jin, Tianbao Yang
Accelerated gradient (AG) methods are breakthroughs in convex optimization, improving the convergence rate of the gradient descent method for optimization with smooth functions. Ho…
First-order Stochastic Algorithms for Escaping From Saddle Points in Almost Linear Time
Yi Xu, Rong Jin, Tianbao Yang
Two classes of methods have been proposed for escaping from saddle points with one using the second-order information carried by the Hessian and the other adding the noise into the…
Homotopy Smoothing for Non-Smooth Problems with Lower Complexity than
Yi Xu, Yan Yan, Qihang Lin +1
In this paper, we develop a novel {\bf ho}moto{\bf p}y {\bf s}moothing (HOPS) algorithm for solving a family of non-smooth problems that is composed of a non-smooth term with an ex…
Accelerate Stochastic Subgradient Method by Leveraging Local Growth Condition
Yi Xu, Qihang Lin, Tianbao Yang
In this paper, a new theory is developed for first-order stochastic convex optimization, showing that the global convergence rate is sufficiently quantified by a local growth rate…