Concentration Inequalities and Moment Bounds for Sample Covariance Operators

Abstract

Let $X,X_1,\dots ,X_n,\dots$ be i.i.d. centered Gaussian random variables in a separable Banach space $E$ with covariance operator $\Sigma :$ $$\Sigma :E^{\ast }\mapsto E,\Sigma u=\mathbb{E}⟨X,u⟩,u\in E^{\ast }.$$ The sample covariance operator $\hat{\Sigma }:E^{\ast }\mapsto E$ is defined as $$\hat{\Sigma }u:=n^{-1}\sum _{j=1}^n⟨X_j,u⟩X_j,u\in E^{\ast }.$$ The goal of the paper is to obtain concentration inequalities and expectation bounds for the operator norm $\Vert \hat{\Sigma }-\Sigma \Vert$ of the deviation of the sample covariance operator from the true covariance operator. In particular, it is shown that $$\mathbb{E}\Vert \hat{\Sigma }-\Sigma \Vert \asymp \Vert \Sigma \Vert (\sqrt{\frac{\mathbf{r}(\Sigma )}{n}}\bigvee \frac{\mathbf{r}(\Sigma )}{n}),$$ where $$\mathbf{r}(\Sigma ):=\frac{(\mathbb{E}\Vert X\Vert {)}^2}{\Vert \Sigma \Vert }.$$ Moreover, under the assumption that $\mathbf{r}(\Sigma )\lesssim n,$ it is proved that, for all $t\ge 1,$ with probability at least $1-e^{-t}$ \begin{align*} \Bigl|\|\hatΣ- Σ\|-{\mathbb E}\|\hatΣ- Σ\|\Bigr| \lesssim \|Σ\|\biggl(\sqrt{\frac{t}{n}}\bigvee \frac{t}{n}\biggr). \end{align*}

References in corpus (1)

• Concentration via chaining method and its applications

Cited by in corpus (10)

• On the spectral norm of Gaussian random matrices
• Asymptotics of Empirical Eigen-structure for Ultra-high Dimensional Spiked Covariance Model
• Selection and Aggregation of Conformal Prediction Sets
• Bootstrap confidence sets for spectral projectors of sample covariance
• PAC-Bayesian bounds for Principal Component Analysis in Hilbert spaces
• The Dynamics of Gradient Descent for Overparametrized Neural Networks
• Gaussian Graphical Model exploration and selection in high dimension low sample size setting
• Convergence Rate of Krasulina Estimator
• Regularized ERM on random subspaces
• Multi-sample estimation of centered log-ratio matrix in microbiome studies