99 citations · 222 across the 29 of their papers we have counts for
4 papers · 2 filters
New Convergence Aspects of Stochastic Gradient Algorithms
Lam M. Nguyen, Phuong Ha Nguyen, Peter Richtárik +3
The classical convergence analysis of SGD is carried out under the assumption that the norm of the stochastic gradient is uniformly bounded. While this might hold for some loss fun…
Tight Dimension Independent Lower Bound on the Expected Convergence Rate for Diminishing Step Sizes in SGD
Phuong Ha Nguyen, Lam M. Nguyen, Marten van Dijk
We study the convergence of Stochastic Gradient Descent (SGD) for strongly convex objective functions. We prove for all a lower bound on the expected convergence rate after the…
Characterization of Convex Objective Functions and Optimal Expected Convergence Rates for SGD
Marten van Dijk, Lam M. Nguyen, Phuong Ha Nguyen +1
We study Stochastic Gradient Descent (SGD) with diminishing step sizes for convex objective functions. We introduce a definitional framework and theory that defines and characteriz…
SGD and Hogwild! Convergence Without the Bounded Gradients Assumption
Lam M. Nguyen, Phuong Ha Nguyen, Marten van Dijk +3
Stochastic gradient descent (SGD) is the optimization algorithm of choice in many machine learning applications such as regularized empirical risk minimization and training deep ne…