3k citations · 24.2k across the 299 of their papers we have counts for
6 papers · 2 filters
A Control-Theoretic Perspective on Optimal High-Order Optimization
Tianyi Lin, Michael. I. Jordan
We provide a control-theoretic perspective on optimal tensor algorithms for minimizing a convex function in a finite-dimensional Euclidean space. Given a function $Φ: \mathbb{R}^d…
First-order methods almost always avoid saddle points: the case of vanishing step-sizes
Ioannis Panageas, Georgios Piliouras, Xiao Wang
In a series of papers \cite{LSJR16, PP17, LPP}, it was established that some of the most commonly used first order methods almost surely (under random initializations) and with ste…
Generalized Momentum-Based Methods: A Hamiltonian Perspective
Jelena Diakonikolas, Michael I. Jordan
We take a Hamiltonian-based perspective to generalize Nesterov's accelerated gradient descent and Polyak's heavy ball method to a broad class of momentum methods in the setting of…
A Dynamical Systems Perspective on Nesterov Acceleration
Michael Muehlebach, Michael I. Jordan
We present a dynamical system framework for understanding Nesterov's accelerated gradient method. In contrast to earlier work, our derivation does not rely on a vanishing step size…
On the Adaptivity of Stochastic Gradient-Based Optimization
Lihua Lei, Michael I. Jordan
Stochastic-gradient-based optimization has been a core enabling methodology in applications to large-scale problems in machine learning and related areas. Despite the progress, the…
Acceleration via Symplectic Discretization of High-Resolution Differential Equations
Bin Shi, Simon S. Du, Weijie J. Su +1
We study first-order optimization methods obtained by discretizing ordinary differential equations (ODEs) corresponding to Nesterov's accelerated gradient methods (NAGs) and Polyak…