8 papers
Accelerated Stochastic Zeroth-Order Quasar-Convex Optimization
Eméric Gbaguidi, Julien Hermant
We consider unconstrained minimization of smooth quasar-convex functions when only noisy function evaluations are accessible through a stochastic zeroth-order oracle. For these non…
Acceleration for Polyak-Åojasiewicz Functions with a Gradient Aiming Condition
Julien Hermant
It is known that when minimizing smooth Polyak-Åojasiewicz (PL) functions, momentum algorithms cannot significantly improve the convergence bound of gradient descent, contrasting…
Gradient correlation is a key ingredient to accelerate SGD with momentum
Julien Hermant, Marien Renaud, Jean-François Aujol +2
Empirically, it has been observed that adding momentum to Stochastic Gradient Descent (SGD) accelerates the convergence of the algorithm. However, the literature has been rather pe…
Continuized Nesterov Momentum Achieves the Complexity in Smooth Nonconvex Optimization
Julien Hermant, Jean-François Aujol, Charles Dossal +3
For first-order optimization of non-convex functions with Lipschitz-continuous gradient and Hessian, the best-known complexity for reaching an -approximation of a stat…
Study of the behaviour of Nesterov Accelerated Gradient in a non convex setting: the strongly quasar convex case
Julien Hermant, Jean-François Aujol, Charles Dossal +1
We study the convergence of Nesterov Accelerated Gradient (NAG) minimization algorithmapplied to a class of non convex functions called strongly quasar convex functions. We show th…
Continuized Nesterov Acceleration for Non-Convex Optimization
Julien Hermant, Jean-François Aujol, Charles Dossal +2
In convex optimization, continuous-time counterparts have been a fruitful tool for analyzing momentum algorithms. Fewer such examples are available when the function to minimize is…