99 citations · 203 across the 21 of their papers we have counts for
6 papers · 1 filter
A Unified Convergence Analysis for Shuffling-Type Gradient Methods
Lam M. Nguyen, Quoc Tran-Dinh, Dzung T. Phan +2
In this paper, we propose a unified convergence analysis for a class of generic shuffling-type gradient methods for solving finite-sum optimization problems. Our analysis works wit…
Finite-Sum Smooth Optimization with SARAH
Lam M. Nguyen, Marten van Dijk, Dzung T. Phan +3
The total complexity (measured as the total number of gradient computations) of a stochastic first-order optimization algorithm that finds a first-order stationary point of a finit…
New Convergence Aspects of Stochastic Gradient Algorithms
Lam M. Nguyen, Phuong Ha Nguyen, Peter Richtárik +3
The classical convergence analysis of SGD is carried out under the assumption that the norm of the stochastic gradient is uniformly bounded. While this might hold for some loss fun…
Tight Dimension Independent Lower Bound on the Expected Convergence Rate for Diminishing Step Sizes in SGD
Phuong Ha Nguyen, Lam M. Nguyen, Marten van Dijk
We study the convergence of Stochastic Gradient Descent (SGD) for strongly convex objective functions. We prove for all a lower bound on the expected convergence rate after the…
Characterization of Convex Objective Functions and Optimal Expected Convergence Rates for SGD
Marten van Dijk, Lam M. Nguyen, Phuong Ha Nguyen +1
We study Stochastic Gradient Descent (SGD) with diminishing step sizes for convex objective functions. We introduce a definitional framework and theory that defines and characteriz…
SGD and Hogwild! Convergence Without the Bounded Gradients Assumption
Lam M. Nguyen, Phuong Ha Nguyen, Marten van Dijk +3
Stochastic gradient descent (SGD) is the optimization algorithm of choice in many machine learning applications such as regularized empirical risk minimization and training deep ne…