209 citations · 415 across the 30 of their papers we have counts for
12 papers · 1 filter
A Model-Based Method for Minimizing CVaR and Beyond
Si Yi Meng, Robert M. Gower
We develop a variant of the stochastic prox-linear method for minimizing the Conditional Value-at-Risk (CVaR) objective. CVaR is a risk measure focused on minimizing worst-case per…
A Bregman-Kaczmarz method for nonlinear systems of equations
Robert Gower, Dirk A. Lorenz, Maximilian Winkler
We propose a new randomized method for solving systems of nonlinear equations, which can find sparse solutions or solutions under certain simple constraints. The scheme only takes…
Handbook of Convergence Theorems for (Stochastic) Gradient Methods
Guillaume Garrigos, Robert M. Gower
This is a handbook of simple proofs of the convergence of gradient and stochastic gradient descent type methods. We consider functions that are Lipschitz, smooth, convex, strongly…
A Stochastic Proximal Polyak Step Size
Fabian Schaipp, Robert M. Gower, Michael Ulbrich
Recently, the stochastic Polyak step size (SPS) has emerged as a competitive adaptive step size scheme for stochastic gradient descent. Here we develop ProxSPS, a proximal variant…
SAN: Stochastic Average Newton Algorithm for Minimizing Finite Sums
Jiabin Chen, Rui Yuan, Guillaume Garrigos +1
We present a principled approach for designing stochastic Newton methods for solving finite sum optimization problems. Our approach has two steps. First, we re-write the stationari…
: A Fast sketching based solver for large scale ridge regression
Nidham Gazagnadou, Mark Ibrahim, Robert M. Gower
We propose new variants of the sketch-and-project method for solving large scale ridge regression problems. Firstly, we propose a new momentum alternative and provide a theorem sho…
SGD for Structured Nonconvex Functions: Learning Rates, Minibatching and Interpolation
Robert M. Gower, Othmane Sebbouh, Nicolas Loizou
Stochastic Gradient Descent (SGD) is being used routinely for optimizing non-convex functions. Yet, the standard convergence theory for SGD in the smooth non-convex setting gives a…
Fast Linear Convergence of Randomized BFGS
Dmitry Kovalev, Robert M. Gower, Peter Richtárik +1
Since the late 1950's when quasi-Newton methods first appeared, they have become one of the most widely used and efficient algorithmic paradigms for unconstrained optimization. Des…
Towards closing the gap between the theory and practice of SVRG
Othmane Sebbouh, Nidham Gazagnadou, Samy Jelassi +2
Among the very first variance reduced stochastic methods for solving the empirical risk minimization problem was the SVRG method (Johnson & Zhang 2013). SVRG is an inner-outer loop…
RSN: Randomized Subspace Newton
Robert M. Gower, Dmitry Kovalev, Felix Lieder +1
We develop a randomized Newton method capable of solving learning problems with huge dimensional feature spaces, which is a common setting in applications such as medical imaging,…
Improving SAGA via a Probabilistic Interpolation with Gradient Descent
Adel Bibi, Alibek Sailanbayev, Bernard Ghanem +2
We develop and analyze a new algorithm for empirical risk minimization, which is the key paradigm for training supervised machine learning models. Our method---SAGD---is based on a…
Action constrained quasi-Newton methods
Robert Mansel Gower, Jacek Gondzio
At the heart of Newton based optimization methods is a sequence of symmetric linear systems. Each consecutive system in this sequence is similar to the next, so solving them separa…