176 citations · 538 across the 62 of their papers we have counts for
4 papers · 1 filter
Stochastic Mirror Descent: Convergence Analysis and Adaptive Variants via the Mirror Stochastic Polyak Stepsize
Ryan D'Orazio, Nicolas Loizou, Issam Laradji +1
We investigate the convergence of stochastic mirror descent (SMD) under interpolation in relatively smooth and smooth convex optimization. In relatively smooth convex optimization…
Stochastic Polyak Step-size for SGD: An Adaptive Learning Rate for Fast Convergence
Nicolas Loizou, Sharan Vaswani, Issam Laradji +1
We propose a stochastic variant of the classical Polyak step-size (Polyak, 1987) commonly used in the subgradient method. Although computing the Polyak step-size requires knowledge…
Let's Make Block Coordinate Descent Converge Faster: Faster Greedy Rules, Message-Passing, Active-Set Complexity, and Superlinear Convergence
Julie Nutini, Issam Laradji, Mark Schmidt
Block coordinate descent (BCD) methods are widely used for large-scale numerical optimization because of their cheap iteration costs, low memory requirements, amenability to parall…
Coordinate Descent Converges Faster with the Gauss-Southwell Rule Than Random Selection
Julie Nutini, Mark Schmidt, Issam H. Laradji +2
There has been significant recent work on the theory and application of randomized coordinate descent algorithms, beginning with the work of Nesterov [SIAM J. Optim., 22(2), 2012],…