An Eigenvector Perturbation Bound and Its Application to Robust Covariance Estimation
arXiv:1603.03516
Abstract
In statistics and machine learning, people are often interested in the eigenvectors (or singular vectors) of certain matrices (e.g. covariance matrices, data matrices, etc). However, those matrices are usually perturbed by noises or statistical errors, either from random sampling or structural patterns. One usually employs Davis-Kahan theorem to bound the difference between the eigenvectors of a matrix and those of a perturbed matrix , in terms of norm. In this paper, we prove that when is a low-rank and incoherent matrix, the norm perturbation bound of singular vectors (or eigenvectors in the symmetric case) is smaller by a factor of or for left and right vectors, where and are the matrix dimensions. The power of this new perturbation result is shown in robust covariance estimation, particularly when random variables have heavy tails. There, we propose new robust covariance estimators and establish their asymptotic properties using the newly developed perturbation bound. Our theoretical results are verified through extensive numerical experiments.
arXiv comment48 pages, 5 figures
References in corpus (6)
- Sparse principal component analysis and iterative thresholding
- Optimal detection of sparse principal components in high dimension
- Incoherence-Optimal Matrix Completion
- Asymptotics of Empirical Eigen-structure for Ultra-high Dimensional Spiked Covariance Model
- Robust Estimation of High-Dimensional Mean Regression
- Rate-Optimal Perturbation Bounds for Singular Subspaces with Applications to High-Dimensional Statistics
Cited by in corpus (6)
- Near-optimal bounds for phase synchronization
- Estimation of the covariance structure of heavy-tailed distributions
- Principal component analysis for big data
- Eigenvector Under Random Perturbation: A Nonasymptotic Rayleigh-Schrödinger Theory
- A Schatten- Low-rank Matrix Perturbation Analysis via Perturbation Projection Error Bound
- Robust Estimation of Covariance Matrices: Adversarial Contamination and Beyond