170 citations · 909 across the 66 of their papers we have counts for
Showing 2018 · stat.MLShow all
2 papers · 2 filters
stat.ML2018★ 170 cited
AdaGrad stepsizes: Sharp convergence over nonconvex landscapes
Rachel Ward, Xiaoxia Wu, Leon Bottou
Adaptive gradient methods such as AdaGrad and its variants update the stepsize in stochastic gradient descent on the fly according to the gradients received along the way; such met…
stat.ML2018★ 55 cited
WNGrad: Learn the Learning Rate in Gradient Descent
Xiaoxia Wu, Rachel Ward, Léon Bottou
Adjusting the learning rate schedule in stochastic gradient methods is an important unresolved problem which requires tuning in practice. If certain parameters of the loss function…