89 citations · 285 across the 12 of their papers we have counts for
8 papers · 1 filter
Gradient Descent and the Power Method: Exploiting their connection to find the leftmost eigen-pair and escape saddle points
Rachael Tappenden, Martin Takáč
This work shows that applying Gradient Descent (GD) with a fixed step size to minimize a (possibly nonconvex) quadratic function is equivalent to running the Power Method (PM) on t…
New Convergence Aspects of Stochastic Gradient Algorithms
Lam M. Nguyen, Phuong Ha Nguyen, Peter Richtárik +3
The classical convergence analysis of SGD is carried out under the assumption that the norm of the stochastic gradient is uniformly bounded. While this might hold for some loss fun…
Inexact SARAH Algorithm for Stochastic Optimization
Lam M. Nguyen, Katya Scheinberg, Martin Takáč
We develop and analyze a variant of the SARAH algorithm, which does not require computation of the exact gradient. Thus this new method can be applied to general expectation minimi…
Randomized sketch descent methods for non-separable linearly constrained optimization
Ion Necoara, Martin Takac
In this paper we consider large-scale smooth optimization problems with multiple linear coupled constraints. Due to the non-separability of the constraints, arbitrary random sketch…
SGD and Hogwild! Convergence Without the Bounded Gradients Assumption
Lam M. Nguyen, Phuong Ha Nguyen, Marten van Dijk +3
Stochastic gradient descent (SGD) is the optimization algorithm of choice in many machine learning applications such as regularized empirical risk minimization and training deep ne…
Dual Free Adaptive Minibatch SDCA for Empirical Risk Minimization
Xi He, Rachael Tappenden, Martin Takac
In this paper we develop an adaptive dual free Stochastic Dual Coordinate Ascent (adfSDCA) algorithm for regularized empirical risk minimization problems. This is motivated by the…