90 citations · 162 across the 13 of their papers we have counts for
Showing 2020Show all
3 papers · 1 filter
cs.CV2020★ 2 cited
Controllable Orthogonalization in Training DNNs
Lei Huang, Li Liu, Fan Zhu +4
Orthogonality is widely used for training deep neural networks (DNNs) due to its ability to maintain all singular values of the Jacobian close to 1 and reduce redundancy in represe…
cs.CV2020
An Investigation into the Stochasticity of Batch Whitening
Lei Huang, Lei Zhao, Yi Zhou +3
Batch Normalization (BN) is extensively employed in various network architectures by performing standardization within mini-batches. A full understanding of the process has been a…
cs.CV2020★ 2 cited
Layer-wise Conditioning Analysis in Exploring the Learning Dynamics of DNNs
Lei Huang, Jie Qin, Li Liu +2
Conditioning analysis uncovers the landscape of an optimization objective by exploring the spectrum of its curvature matrix. This has been well explored theoretically for linear mo…