Eigenvalue distribution of nonlinear models of random matrices
arXiv:1904.03090 · doi:10.1214/21-EJP699
Abstract
This paper is concerned with the asymptotic empirical eigenvalue distribution of a non linear random matrix ensemble. More precisely we consider with where and are random rectangular matrices with i.i.d. centered entries. The function is applied pointwise and can be seen as an activation function in (random) neural networks. We compute the asymptotic empirical distribution of this ensemble in the case where and have sub-Gaussian tails and is real analytic. This extends a previous result where the case of Gaussian matrices and is considered. We also investigate the same questions in the multi-layer case, regarding neural network applications.
arXiv comment36 pages, 19 figures. Paper shortened (the behavior of the largest eigenvalue is removed and postponed to another article after noticing an error)
References in corpus (18)
- Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift
- Deep Learning in Neural Networks: An Overview
- Google's Neural Machine Translation System: Bridging the Gap between Human and Machine Translation
- Machine learning and the physical sciences
- The Loss Surfaces of Multilayer Networks
- Universality at the edge of the spectrum in Wigner random matrices
- Products of Rectangular Random Matrices: Singular Values and Progressive Scattering
- Deep Neural Networks with Random Gaussian Weights: A Universal Classification Strategy?
- Entropy and mutual information in models of deep neural networks
- The spectrum of kernel random matrices
- Product of Ginibre matrices: Fuss-Catalan and Raney distributions
- Singular values of products of Ginibre random matrices, multiple orthogonal polynomials and hard edge scaling limits
- Universal microscopic correlation functions for products of independent Ginibre matrices
- Products of Many Large Random Matrices and Gradients in Deep Neural Networks
- On the Selection of Initialization and Activation Function for Deep Neural Networks
- Raney distributions and random matrix theory
- Concentration of Measure and Large Random Matrices with an application to Sample Covariance Matrices
- Spectral density of products of Wishart dilute random matrices. Part I: the dense case
Cited by in corpus (4)
- Right large deviation principle for the top eigenvalue of the sum or product of invariant random matrices
- Structured random matrices and cyclic cumulants: A free probability approach
- Largest Eigenvalues of the Conjugate Kernel of Single-Layered Neural Networks
- Distribution of singular values in large sample cross-covariance matrices