Normal approximation and concentration of spectral projectors of sample covariance

Abstract

Let $X,X_1,\dots ,X_n$ be i.i.d. Gaussian random variables in a separable Hilbert space $\mathbb{H}$ with zero mean and covariance operator $\Sigma =\mathbb{E}(X\otimes X),$ and let $\hat{\Sigma }:=n^{-1}{\sum }_{j=1}^n(X_j\otimes X_j)$ be the sample (empirical) covariance operator based on $(X_1,\dots ,X_n).$ Denote by $P_r$ the spectral projector of $\Sigma$ corresponding to its $r$-th eigenvalue ${\mu }_r$ and by ${\hat{P}}_r$ the empirical counterpart of $P_r.$ The main goal of the paper is to obtain tight bounds on $$\underset{x\in \mathbb{R}}{\sup }\left|\mathbb{P}\left\{\frac{\Vert {\hat{P}}_r-P_r{\Vert }_2^2-\mathbb{E}\Vert {\hat{P}}_r-P_r{\Vert }_2^2}{{\mathrm{Var}}^{1/2}(\Vert {\hat{P}}_r-P_r{\Vert }_2^2)}\le x\right\}-\Phi (x)\right|,$$ where $\Vert \cdot {\Vert }_2$ denotes the Hilbert--Schmidt norm and $\Phi$ is the standard normal distribution function. Such accuracy of normal approximation of the distribution of squared Hilbert--Schmidt error is characterized in terms of so called effective rank of $\Sigma$ defined as $\mathbf{r}(\Sigma )=\frac{\mathrm{tr}(\Sigma )}{\Vert \Sigma {\Vert }_{\infty }},$ where $\mathrm{tr}(\Sigma )$ is the trace of $\Sigma$ and $\Vert \Sigma {\Vert }_{\infty }$ is its operator norm, as well as another parameter characterizing the size of $\mathrm{Var}(\Vert {\hat{P}}_r-P_r{\Vert }_2^2).$ Other results include non-asymptotic bounds and asymptotic representations for the mean squared Hilbert--Schmidt norm error $\mathbb{E}\Vert {\hat{P}}_r-P_r{\Vert }_2^2$ and the variance $\mathrm{Var}(\Vert {\hat{P}}_r-P_r{\Vert }_2^2),$ and concentration inequalities for $\Vert {\hat{P}}_r-P_r{\Vert }_2^2$ around its expectation.

arXiv commentarXiv admin note: text overlap with arXiv:1408.4643

References in corpus (1)

• Sparse PCA: Optimal rates and adaptive estimation

Cited by in corpus (23)

• Entrywise Estimation of Singular Vectors of Low-Rank Matrices with Heteroskedasticity and Dependence
• Singular vector and singular subspace distribution for the matrix denoising model
• De-biased sparse PCA: Inference and testing for eigenstructure of large covariance matrices
• Fast Robust Subspace Tracking via PCA in Sparse Data-Dependent Noise
• On the number of principal components in high dimensions
• Dimensionality reduction, regularization, and generalization in overparameterized regressions
• Hypothesis testing for eigenspaces of covariance matrix
• PAC-Bayesian bounds for Principal Component Analysis in Hilbert spaces
• Normal Approximation and Confidence Region of Singular Subspaces
• Bayesian inference for spectral projectors of the covariance matrix
• Confidence Region of Singular Subspaces for Low-rank Matrix Regression
• Minimax Estimation of Linear Functions of Eigenvectors in the Face of Small Eigen-Gaps
• Estimating covariance and precision matrices along subspaces
• Kernel Autocovariance Operators of Stationary Processes: Estimation and Convergence
• Recovery of spectrum from estimated covariance matrices and statistical kernels for machine learning and big data
• Bayesian Inverse Problems with Heterogeneous Variance
• All of Linear Regression
• Multi-sample estimation of centered log-ratio matrix in microbiome studies
• Moment bounds for large autocovariance matrices under dependence
• Convergence Rate of Krasulina Estimator
• Finite sample Bernstein-von Mises theorems for functionals and spectral projectors of the covariance matrix
• An Analytical Formula for Spectrum Reconstruction
• Robust Principal Component Analysis in Hilbert spaces