Large-Dimensional Random Matrix Theory and Its Applications in Deep Learning and Wireless Communications

Abstract

Large-dimensional random matrix theory, RMT for short, which originates from the research field of quantum physics, has shown tremendous capability in providing deep insights into large dimensional systems. With the fact that we have entered an unprecedented era full of massive amounts of data and large complex systems, RMT is expected to play more important roles in the analysis and design of modern systems. In this paper, we review the key results of RMT and its applications in two emerging fields: wireless communications and deep learning. In wireless communications, we show that RMT can be exploited to design the spectrum sensing algorithms for cognitive radio systems and to perform the design and asymptotic analysis for large communication systems. In deep learning, RMT can be utilized to analyze the Hessian, input-output Jacobian and data covariance matrix of the deep neural networks, thereby to understand and improve the convergence and the learning speed of the neural networks. Finally, we highlight some challenges and opportunities in applying RMT to the practical large dimensional systems.

References in corpus (26)

• Deep Learning in Neural Networks: An Overview
• Recurrent Neural Network Regularization
• Reconciling modern machine learning practice and the bias-variance trade-off
• Neural Tangent Kernel: Convergence and Generalization in Neural Networks
• Understanding deep learning requires rethinking generalization
• Exact solutions to the nonlinear dynamics of learning in deep linear neural networks
• Massive Connectivity with Massive MIMO-Part I: Device Activity Detection and Channel Estimation
• The Loss Surfaces of Multilayer Networks
• Learning Overparameterized Neural Networks via Stochastic Gradient Descent on Structured Data
• The generalization error of random features regression: Precise asymptotics and double descent curve
• Cooperative Spectrum Sensing based on the Limiting Eigenvalue Ratio Distribution in Wishart Matrices
• Scaling description of generalization with number of parameters in deep learning
• Limit theorems for sample eigenvalues in a generalized spiked population model
• Massive Connectivity with Massive MIMO-Part II: Achievable Rate Characterization
• The Emergence of Spectral Universality in Deep Networks
• The Distribution of the Largest Eigenvalue in the Gaussian Ensembles
• The largest eigenvalues of sample covariance matrices for a spiked population: diagonal case
• A Fine-Grained Spectral Perspective on Neural Networks
• On Random Matrices Arising in Deep Neural Networks. Gaussian Case
• The Asymptotic Performance of Linear Echo State Neural Networks
• Spectra of the Conjugate Kernel and Neural Tangent Kernel for linear-width neural networks
• A Random Matrix Perspective on Mixtures of Nonlinearities for Deep Learning
• Spectral density of products of Wishart dilute random matrices. Part I: the dense case
• Disentangling Trainability and Generalization in Deep Neural Networks
• Massive Access for 5G and Beyond
• Beyond Random Matrix Theory for Deep Networks