13 citations · 29 across the 6 of their papers we have counts for
Showing 2020Show all
2 papers · 1 filter
cs.LG2020★ 3 cited
Implicit bias of deep linear networks in the large learning rate phase
Wei Huang, Weitao Du, Richard Yi Da Xu +1
Most theoretical studies explaining the regularization effect in deep learning have only focused on gradient descent with a sufficient small learning rate or even gradient flow (in…
cs.LG2020★ 13 cited
On the Neural Tangent Kernel of Deep Networks with Orthogonal Initialization
Wei Huang, Weitao Du, Richard Yi Da Xu
The prevailing thinking is that orthogonal weights are crucial to enforcing dynamical isometry and speeding up training. The increase in learning speed that results from orthogonal…