8 papers
Adaptive Matrix Online Learning through Smoothing with Guarantees for Nonsmooth Nonconvex Optimization
Ruichen Jiang, Zakaria Mhammedi, Mehryar Mohri +1
We study online linear optimization with matrix variables constrained by the operator norm, a setting where the geometry renders designing data-dependent and efficient adaptive alg…
Improving Online-to-Nonconvex Conversion for Smooth Optimization via Double Optimism
Francisco Patitucci, Ruichen Jiang, Aryan Mokhtari
A recent breakthrough in nonconvex optimization is the online-to-nonconvex conversion framework of [Cutkosky et al., 2023], which reformulates the task of finding an -…
On the Complexity of Finding Stationary Points in Nonconvex Simple Bilevel Optimization
Jincheng Cao, Ruichen Jiang, Erfan Yazdandoost Hamedani +1
In this paper, we study the problem of solving a simple bilevel optimization problem, where the upper-level objective is minimized over the solution set of the lower-level problem.…
Non-asymptotic Global Convergence Rates of BFGS with Exact Line Search
Qiujiang Jin, Ruichen Jiang, Aryan Mokhtari
In this paper, we explore the non-asymptotic global convergence rates of the Broyden-Fletcher-Goldfarb-Shanno (BFGS) method implemented with exact line search. Notably, due to Dixo…
Online Learning-guided Learning Rate Adaptation via Gradient Alignment
Ruichen Jiang, Ali Kavis, Aryan Mokhtari
The performance of an optimizer on large-scale deep learning models depends critically on fine-tuning the learning rate, often requiring an extensive grid search over base learning…
Provable Complexity Improvement of AdaGrad over SGD: Upper and Lower Bounds in Stochastic Non-Convex Optimization
Ruichen Jiang, Devyani Maladkar, Aryan Mokhtari
Adaptive gradient methods, such as AdaGrad, are among the most successful optimization algorithms for neural network training. While these methods are known to achieve better dimen…