69 citations · 81 across the 2 of their papers we have counts for
2 papers
cs.LG2017★ 69 cited
Resurrecting the sigmoid in deep learning through dynamical isometry: theory and practice
Jeffrey Pennington, Samuel S. Schoenholz, Surya Ganguli
It is well known that the initialization of weights in deep neural networks can have a dramatic impact on learning speed. For example, ensuring the mean squared singular value of a…
stat.ML2017★ 12 cited
A Correspondence Between Random Neural Networks and Statistical Field Theory
Samuel S. Schoenholz, Jeffrey Pennington, Jascha Sohl-Dickstein
A number of recent papers have provided evidence that practical design questions about neural networks may be tackled theoretically by studying the behavior of random networks. How…