Asymptotic performance of PCA for high-dimensional heteroscedastic data

Abstract

Principal Component Analysis (PCA) is a classical method for reducing the dimensionality of data by projecting them onto a subspace that captures most of their variation. Effective use of PCA in modern applications requires understanding its performance for data that are both high-dimensional and heteroscedastic. This paper analyzes the statistical performance of PCA in this setting, i.e., for high-dimensional data drawn from a low-dimensional subspace and degraded by heteroscedastic noise. We provide simplified expressions for the asymptotic PCA recovery of the underlying subspace, subspace amplitudes and subspace coefficients; the expressions enable both easy and efficient calculation and reasoning about the performance of PCA. We exploit the structure of these expressions to show that, for a fixed average noise variance, the asymptotic recovery of PCA for heteroscedastic data is always worse than that for homoscedastic data (i.e., for noise variances that are equal across samples). Hence, while average noise variance is often a practically convenient measure for the overall quality of data, it gives an overly optimistic estimate of the performance of PCA for heteroscedastic data.

arXiv comment34 pages (including supplement), 17 figures

References in corpus (13)

• Covariance regularization by thresholding
• Rank-Sparsity Incoherence for Matrix Decomposition
• Correcting systematic effects in a large set of photometric lightcurves
• Matrix estimation by Universal Singular Value Thresholding
• Operator norm consistent estimation of large-dimensional sparse covariance matrices
• Finite sample approximation results for principal component analysis: a matrix perturbation approach
• OptShrink: An algorithm for improved low-rank signal matrix denoising by optimal, data-driven singular value shrinkage
• Limit theorems for sample eigenvalues in a generalized spiked population model
• Robust computation of linear models by convex relaxation
• Asymptotic performance of PCA for high-dimensional heteroscedastic data
• Towards a Theoretical Analysis of PCA for Heteroscedastic Data
• Online (and Offline) Robust PCA: Novel Algorithms and Performance Guarantees
• PCA from noisy, linearly reduced data: the diagonal case

Cited by in corpus (8)

• Asymptotic performance of PCA for high-dimensional heteroscedastic data
• ScreeNOT: Exact MSE-Optimal Singular Value Thresholding in Correlated Noise
• HePPCAT: Probabilistic PCA for Data with Heteroscedastic Noise
• Biwhitening Reveals the Rank of a Count Matrix
• Optimal spectral shrinkage and PCA with heteroscedastic noise
• Distributed linear regression by averaging
• Matrix Denoising with Partial Noise Statistics: Optimal Singular Value Shrinkage of Spiked F-Matrices
• A Change-Point Based Control Chart for Detecting Sparse Changes in High-Dimensional Heteroscedastic Data