6 citations · 20 across the 9 of their papers we have counts for
Showing 2021Show all
3 papers · 1 filter
cs.LG2021★ 1 cited
Bandwidth-based Step-Sizes for Non-Convex Stochastic Optimization
Xiaoyu Wang, Mikael Johansson
Many popular learning-rate schedules for deep neural networks combine a decaying trend with local perturbations that attempt to escape saddle points and bad local minima. We derive…
math.OC2021★ 4 cited
On the Convergence of Step Decay Step-Size for Stochastic Optimization
Xiaoyu Wang, Sindri Magnússon, Mikael Johansson
The convergence of stochastic gradient descent is highly dependent on the step-size, especially on non-convex problems such as neural network training. Step decay step-size schedul…
math.OC2021★ 5 cited
On the Convergence of Stochastic Gradient Descent with Bandwidth-based Step Size
Xiaoyu Wang, Ya-xiang Yuan
We investigate the stochastic gradient descent (SGD) method where the step size lies within a banded region instead of being given by a fixed formula. The optimal convergence rate…