163 citations · 496 across the 40 of their papers we have counts for
5 papers · 1 filter
Stochastic Hamiltonian Gradient Methods for Smooth Games
Nicolas Loizou, Hugo Berard, Alexia Jolicoeur-Martineau +3
The success of adversarial formulations in machine learning has brought renewed motivation for smooth games. In this work, we focus on the class of stochastic Hamiltonian methods a…
Unified Analysis of Stochastic Gradient Methods for Composite Convex and Smooth Optimization
Ahmed Khaled, Othmane Sebbouh, Nicolas Loizou +2
We present a unified theorem for the convergence analysis of stochastic gradient algorithms for minimizing a smooth and convex loss plus a convex regularizer. We do this by extendi…
SGD for Structured Nonconvex Functions: Learning Rates, Minibatching and Interpolation
Robert M. Gower, Othmane Sebbouh, Nicolas Loizou
Stochastic Gradient Descent (SGD) is being used routinely for optimizing non-convex functions. Yet, the standard convergence theory for SGD in the smooth non-convex setting gives a…
A Unified Theory of Decentralized SGD with Changing Topology and Local Updates
Anastasia Koloskova, Nicolas Loizou, Sadra Boreiri +2
Decentralized stochastic optimization methods have gained a lot of attention recently, mainly because of their cheap per iteration cost, data locality, and their communication-effi…
Stochastic Polyak Step-size for SGD: An Adaptive Learning Rate for Fast Convergence
Nicolas Loizou, Sharan Vaswani, Issam Laradji +1
We propose a stochastic variant of the classical Polyak step-size (Polyak, 1987) commonly used in the subgradient method. Although computing the Polyak step-size requires knowledge…