11 papers
Stochastic Dynamic Barrier Perturbed Gradient Methods for Nonconvex Simple Bilevel Optimization
Mohammad Mahdi Ahmadi, Jincheng Cao, Aryan Mokhtari +1
We study stochastic simple bilevel optimization with smooth, possibly nonconvex upper- and lower-level objectives accessed only through stochastic gradient oracles. A key challenge…
Adaptive Optimization via Momentum on Variance-Normalized Gradients
Francisco Patitucci, Aryan Mokhtari
We introduce MVN-Grad (Momentum on Variance-Normalized Gradients), an Adam-style optimizer that improves stability and performance by combining two complementary ideas: variance-ba…
Adaptive Matrix Online Learning through Smoothing with Guarantees for Nonsmooth Nonconvex Optimization
Ruichen Jiang, Zakaria Mhammedi, Mehryar Mohri +1
We study online linear optimization with matrix variables constrained by the operator norm, a setting where the geometry renders designing data-dependent and efficient adaptive alg…
Improving Online-to-Nonconvex Conversion for Smooth Optimization via Double Optimism
Francisco Patitucci, Ruichen Jiang, Aryan Mokhtari
A recent breakthrough in nonconvex optimization is the online-to-nonconvex conversion framework of [Cutkosky et al., 2023], which reformulates the task of finding an -…
Affine-Invariant Global Non-Asymptotic Convergence Analysis of BFGS under Self-Concordance
Qiujiang Jin, Aryan Mokhtari
In this paper, we establish global non-asymptotic convergence guarantees for the BFGS quasi-Newton method without requiring strong convexity or the Lipschitz continuity of the grad…
On the Complexity of Finding Stationary Points in Nonconvex Simple Bilevel Optimization
Jincheng Cao, Ruichen Jiang, Erfan Yazdandoost Hamedani +1
In this paper, we study the problem of solving a simple bilevel optimization problem, where the upper-level objective is minimized over the solution set of the lower-level problem.…