Eigenvalue Distribution of Large Random Matrices Arising in Deep Neural Networks: Orthogonal Case

Leonid Pastur

The paper deals with the distribution of singular values of the input-output Jacobian of deep untrained neural networks in the limit of their infinite width. The Jacobian is the pr…