8 citations · 11 across the 2 of their papers we have counts for
2 papers
cs.LG2021★ 8 cited
Strength of Minibatch Noise in SGD
Liu Ziyin, Kangqiao Liu, Takashi Mori +1
The noise in stochastic gradient descent (SGD), caused by minibatch sampling, is poorly understood despite its practical importance in deep learning. This work presents the first s…
stat.ML2020★ 3 cited
Noise and Fluctuation of Finite Learning Rate Stochastic Gradient Descent
Kangqiao Liu, Liu Ziyin, Masahito Ueda
In the vanishing learning rate regime, stochastic gradient descent (SGD) is now relatively well understood. In this work, we propose to study the basic properties of SGD and its va…