239 citations · 1.1k across the 35 of their papers we have counts for
4 papers · 1 filter
Products of Many Large Random Matrices and Gradients in Deep Neural Networks
Boris Hanin, Mihai Nica
We study products of random matrices in the regime where the number of terms and the size of the matrices simultaneously tend to infinity. Our main theorem is that the logarithm of…
The lemniscate tree of a random polynomial
Michael Epstein, Boris Hanin, Erik Lundberg
To each generic complex polynomial there is associated a labeled binary tree (here referred to as a "lemniscate tree") that encodes the topological type of the graph of $|p(…
How to Start Training: The Effect of Initialization and Architecture
Boris Hanin, David Rolnick
We identify and study two common failure modes for early training in deep ReLU nets. For each we give a rigorous proof of when it occurs and how to avoid it, for fully connected an…
Which Neural Net Architectures Give Rise To Exploding and Vanishing Gradients?
Boris Hanin
We give a rigorous analysis of the statistical behavior of gradients in a randomly initialized fully connected network N with ReLU activations. Our results show that the empirical…