500 citations · 1.3k across the 52 of their papers we have counts for
15 papers · 1 filter
New Convergence Aspects of Stochastic Gradient Algorithms
Lam M. Nguyen, Phuong Ha Nguyen, Peter Richtárik +3
The classical convergence analysis of SGD is carried out under the assumption that the norm of the stochastic gradient is uniformly bounded. While this might hold for some loss fun…
A Stochastic Penalty Model for Convex and Nonconvex Optimization with Big Constraints
Konstantin Mishchenko, Peter Richtárik
The last decade witnessed a rise in the importance of supervised learning applications involving {\em big data} and {\em big models}. Big data refers to situations where the amount…
SEGA: Variance Reduction via Gradient Sketching
Filip Hanzely, Konstantin Mishchenko, Peter Richtarik
We propose a randomized first order optimization method--SEGA (SkEtched GrAdient method)-- which progressively throughout its iterations builds a variance-reduced estimate of the g…
Accelerated Coordinate Descent with Arbitrary Sampling and Best Rates for Minibatches
Filip Hanzely, Peter Richtárik
Accelerated coordinate descent is a widely popular optimization algorithm due to its efficiency on large-dimensional problems. It achieves state-of-the-art complexity on an importa…
Nonconvex Variance Reduced Optimization with Arbitrary Sampling
Samuel Horváth, Peter Richtárik
We provide the first importance sampling variants of variance reduced algorithms for empirical risk minimization with non-convex loss functions. In particular, we analyze non-conve…
Improving SAGA via a Probabilistic Interpolation with Gradient Descent
Adel Bibi, Alibek Sailanbayev, Bernard Ghanem +2
We develop and analyze a new algorithm for empirical risk minimization, which is the key paradigm for training supervised machine learning models. Our method---SAGD---is based on a…