The spectrum of kernel random matrices

Abstract

We place ourselves in the setting of high-dimensional statistical inference where the number of variables $p$ in a dataset of interest is of the same order of magnitude as the number of observations $n$. We consider the spectrum of certain kernel random matrices, in particular $n\times n$ matrices whose $(i,j)$th entry is $f(X_i^{\prime }{X}_j/p)$ or $f(\Vert X_i-X_j{\Vert }^2/p)$ where $p$ is the dimension of the data, and $X_i$ are independent data vectors. Here $f$ is assumed to be a locally smooth function. The study is motivated by questions arising in statistics and computer science where these matrices are used to perform, among other things, nonlinear versions of principal component analysis. Surprisingly, we show that in high-dimensions, and for the models we analyze, the problem becomes essentially linear--which is at odds with heuristics sometimes used to justify the usage of these methods. The analysis also highlights certain peculiarities of models widely studied in random matrix theory and raises some questions about their relevance as tools to model high-dimensional data encountered in practice.

Published in at http://dx.doi.org/10.1214/08-AOS648 the Annals of Statistics (http://www.imstat.org/aos/) by the Institute of Mathematical Statistics (http://www.imstat.org)

References in corpus (3)

• Operator norm consistent estimation of large-dimensional sparse covariance matrices
• Concentration of measure and spectra of random matrices: Applications to correlation matrices, elliptical distributions and beyond
• On asymptotics of eigenvectors of large sample covariance matrix

Cited by in corpus (16)

• Concentration of measure and spectra of random matrices: Applications to correlation matrices, elliptical distributions and beyond
• Just Interpolate: Kernel "Ridgeless" Regression Can Generalize
• Learning curves of generic features maps for realistic datasets with a teacher-student model
• A Large Dimensional Analysis of Least Squares Support Vector Machines
• On information plus noise kernel random matrices
• The Quantum Path Kernel: a Generalized Quantum Neural Tangent Kernel for Deep Quantum Machine Learning
• Large Dimensional Analysis of Robust M-Estimators of Covariance with Outliers
• Gaussian Universality of Perceptrons with Random Labels
• Eigenvalue distribution of nonlinear models of random matrices
• On the numerical rank of radial basis function kernels in high dimension
• Feature augmentation for the inversion of the Fourier transform with limited data
• Random features and polynomial rules
• Risk Convergence of Centered Kernel Ridge Regression with Large Dimensional Data
• On the Spectrum of Multi-Space Euclidean Random Matrices
• The High-Dimensional Asymptotics of Principal Component Regression
• Spectral properties of kernel matrices in the flat limit