5 papers · 1 filter
: A Fast sketching based solver for large scale ridge regression
Nidham Gazagnadou, Mark Ibrahim, Robert M. Gower
We propose new variants of the sketch-and-project method for solving large scale ridge regression problems. Firstly, we propose a new momentum alternative and provide a theorem sho…
SGD for Structured Nonconvex Functions: Learning Rates, Minibatching and Interpolation
Robert M. Gower, Othmane Sebbouh, Nicolas Loizou
Stochastic Gradient Descent (SGD) is being used routinely for optimizing non-convex functions. Yet, the standard convergence theory for SGD in the smooth non-convex setting gives a…
Fast Linear Convergence of Randomized BFGS
Dmitry Kovalev, Robert M. Gower, Peter Richtárik +1
Since the late 1950's when quasi-Newton methods first appeared, they have become one of the most widely used and efficient algorithmic paradigms for unconstrained optimization. Des…
RSN: Randomized Subspace Newton
Robert M. Gower, Dmitry Kovalev, Felix Lieder +1
We develop a randomized Newton method capable of solving learning problems with huge dimensional feature spaces, which is a common setting in applications such as medical imaging,…
Improving SAGA via a Probabilistic Interpolation with Gradient Descent
Adel Bibi, Alibek Sailanbayev, Bernard Ghanem +2
We develop and analyze a new algorithm for empirical risk minimization, which is the key paradigm for training supervised machine learning models. Our method---SAGD---is based on a…