42 citations · 50 across the 2 of their papers we have counts for
3 papers
stat.ML2020★ 8 cited
Statistical Adaptive Stochastic Gradient Methods
Pengchuan Zhang, Hunter Lang, Qiang Liu +1
We propose a statistical adaptive procedure called SALSA for automatically scheduling the learning rate (step size) in stochastic gradient methods. SALSA first uses a smoothed stoc…
cs.LG2019★ 42 cited
Understanding the Role of Momentum in Stochastic Gradient Methods
Igor Gitman, Hunter Lang, Pengchuan Zhang +1
The use of momentum in stochastic gradient methods has become a widespread practice in machine learning. Different variants of momentum, including heavy-ball momentum, Nesterov's a…
stat.ML2019
Using Statistics to Automate Stochastic Optimization
Hunter Lang, Pengchuan Zhang, Lin Xiao
Despite the development of numerous adaptive optimizers, tuning the learning rate of stochastic gradient methods remains a major roadblock to obtaining good practical performance i…