4 papers
Near-Optimal Nonconvex-Strongly-Convex Bilevel Optimization with Fully First-Order Oracles
Lesi Chen, Yaohua Ma, Jingzhao Zhang
In this work, we consider bilevel optimization when the lower-level problem is strongly convex. Recent works show that with a Hessian-vector product (HVP) oracle, one can provably…
On Finding Small Hyper-Gradients in Bilevel Optimization: Hardness Results and Improved Analysis
Lesi Chen, Jing Xu, Jingzhao Zhang
Bilevel optimization reveals the inner structure of otherwise oblique optimization problems, such as hyperparameter tuning, neural architecture search, and meta-learning. A common…
Faster Stochastic Algorithms for Minimax Optimization under Polyak--Åojasiewicz Conditions
Lesi Chen, Boyuan Yao, Luo Luo
This paper considers stochastic first-order algorithms for minimax optimization under Polyak--Åojasiewicz (PL) conditions. We propose SPIDER-GDA for solving the finite-sum problem…
Near-Optimal Algorithms for Making the Gradient Small in Stochastic Minimax Optimization
Lesi Chen, Luo Luo
We study the problem of finding a near-stationary point for smooth minimax optimization. The recently proposed extra anchored gradient (EAG) methods achieve the optimal convergence…