30 citations · 69 across the 9 of their papers we have counts for
Showing 2020Show all
3 papers · 1 filter
stat.ML2020★ 6 cited
Spherical Motion Dynamics: Learning Dynamics of Neural Network with Normalization, Weight Decay, and SGD
Ruosi Wan, Zhanxing Zhu, Xiangyu Zhang +1
In this work, we comprehensively reveal the learning dynamics of neural network with normalization, weight decay (WD), and SGD (with momentum), named as Spherical Motion Dynamics (…
cs.NE2020★ 16 cited
Angle-based Search Space Shrinking for Neural Architecture Search
Yiming Hu, Yuding Liang, Zichao Guo +5
In this work, we present a simple and general search space shrinking method, called Angle-Based search space Shrinking (ABS), for Neural Architecture Search (NAS). Our approach pro…
cs.CV2020★ 30 cited
Towards Stabilizing Batch Statistics in Backward Propagation of Batch Normalization
Junjie Yan, Ruosi Wan, Xiangyu Zhang +3
Batch Normalization (BN) is one of the most widely used techniques in Deep Learning field. But its performance can awfully degrade with insufficient batch size. This weakness limit…