75 citations · 131 across the 20 of their papers we have counts for
Showing math.OCShow all
2 papers · 1 filter
math.OC2024
(Accelerated) Noise-adaptive Stochastic Heavy-Ball Momentum
Anh Dang, Reza Babanezhad, Sharan Vaswani
Stochastic heavy ball momentum (SHB) is commonly used to train machine learning models, and often provides empirical improvements over stochastic gradient descent. By primarily foc…
math.OC2021★ 1 cited
Towards Noise-adaptive, Problem-adaptive (Accelerated) Stochastic Gradient Descent
Sharan Vaswani, Benjamin Dubois-Taine, Reza Babanezhad
We aim to make stochastic gradient descent (SGD) adaptive to (i) the noise in the stochastic gradients and (ii) problem-dependent constants. When minimizing smooth, strongly-…