50 citations · 196 across the 15 of their papers we have counts for
Showing 2019Show all
2 papers · 1 filter
cs.LG2019★ 42 cited
Understanding the Role of Momentum in Stochastic Gradient Methods
Igor Gitman, Hunter Lang, Pengchuan Zhang +1
The use of momentum in stochastic gradient methods has become a widespread practice in machine learning. Different variants of momentum, including heavy-ball momentum, Nesterov's a…
stat.ML2019★ 9 cited
Using Statistics to Automate Stochastic Optimization
Hunter Lang, Pengchuan Zhang, Lin Xiao
Despite the development of numerous adaptive optimizers, tuning the learning rate of stochastic gradient methods remains a major roadblock to obtaining good practical performance i…