167 citations · 181 across the 12 of their papers we have counts for
Showing 2022Show all
2 papers · 1 filter
cs.LG2022★ 2 cited
Dynamical Isometry for Residual Networks
Advait Gadhikar, Rebekka Burkholz
The training success, training speed and generalization ability of neural networks rely crucially on the choice of random parameter initialization. It has been shown for multiple a…
cs.LG2022★ 4 cited
Why Random Pruning Is All We Need to Start Sparse
Advait Gadhikar, Sohom Mukherjee, Rebekka Burkholz
Random masks define surprisingly effective sparse neural network models, as has been shown empirically. The resulting sparse networks can often compete with dense architectures and…