3 citations · 3 across the 2 of their papers we have counts for
4 papers
Unified Analysis of Stochastic Gradient Methods for Composite Convex and Smooth Optimization
Ahmed Khaled, Othmane Sebbouh, Nicolas Loizou +2
We present a unified theorem for the convergence analysis of stochastic gradient algorithms for minimizing a smooth and convex loss plus a convex regularizer. We do this by extendi…
SGD for Structured Nonconvex Functions: Learning Rates, Minibatching and Interpolation
Robert M. Gower, Othmane Sebbouh, Nicolas Loizou
Stochastic Gradient Descent (SGD) is being used routinely for optimizing non-convex functions. Yet, the standard convergence theory for SGD in the smooth non-convex setting gives a…
Almost sure convergence rates for Stochastic Gradient Descent and Stochastic Heavy Ball
Othmane Sebbouh, Robert M. Gower, Aaron Defazio
We study stochastic gradient descent (SGD) and the stochastic heavy ball method (SHB, otherwise known as the momentum method) for the general stochastic approximation problem. For…
Nesterov's acceleration and Polyak's heavy ball method in continuous time: convergence rate analysis under geometric conditions and perturbations
Othmane Sebbouh, Charles Dossal, Aude Rondepierre
In this article a family of second order ODEs associated to inertial gradient descend is studied. These ODEs are widely used to build trajectories converging to a minimizer o…