4 papers · 1 filter
Near-Optimal Nonconvex-Strongly-Convex Bilevel Optimization with Fully First-Order Oracles
Lesi Chen, Yaohua Ma, Jingzhao Zhang
In this work, we consider bilevel optimization when the lower-level problem is strongly convex. Recent works show that with a Hessian-vector product (HVP) oracle, one can provably…
On Finding Small Hyper-Gradients in Bilevel Optimization: Hardness Results and Improved Analysis
Lesi Chen, Jing Xu, Jingzhao Zhang
Bilevel optimization reveals the inner structure of otherwise oblique optimization problems, such as hyperparameter tuning, neural architecture search, and meta-learning. A common…
Faster Stochastic Algorithms for Minimax Optimization under Polyak--Åojasiewicz Conditions
Lesi Chen, Boyuan Yao, Luo Luo
This paper considers stochastic first-order algorithms for minimax optimization under Polyak--Åojasiewicz (PL) conditions. We propose SPIDER-GDA for solving the finite-sum problem…
Faster Gradient-Free Algorithms for Nonsmooth Nonconvex Stochastic Optimization
Lesi Chen, Jing Xu, Luo Luo
We consider the optimization problem of the form , where the component is -mean-squared Lipschitz b…