932 citations · 2.2k across the 38 of their papers we have counts for
Showing 2019Show all
2 papers · 1 filter
cs.LG2019★ 10 cited
How noise affects the Hessian spectrum in overparameterized neural networks
Mingwei Wei, David J Schwab
Stochastic gradient descent (SGD) forms the core optimization method for deep neural networks. While some theoretical progress has been made, it still remains unclear why SGD leads…
cs.LG2019
Mean-field Analysis of Batch Normalization
Mingwei Wei, James Stokes, David J Schwab
Batch Normalization (BatchNorm) is an extremely useful component of modern neural network architectures, enabling optimization using higher learning rates and achieving faster conv…