21 citations · 49 across the 12 of their papers we have counts for
Showing 2019Show all
2 papers · 1 filter
cs.LG2019★ 11 cited
Positively Scale-Invariant Flatness of ReLU Neural Networks
Mingyang Yi, Qi Meng, Wei Chen +2
It was empirically confirmed by Keskar et al.\cite{SharpMinima} that flatter minima generalize better. However, for the popular ReLU network, sharp minimum can also generalize well…
cs.LG2019
Stabilize Deep ResNet with A Sharp Scaling Factor
Huishuai Zhang, Da Yu, Mingyang Yi +2
We study the stability and convergence of training deep ResNets with gradient descent. Specifically, we show that the parametric branch in the residual block should be scaled down…