Optimal spectral shrinkage and PCA with heteroscedastic noise

Abstract

This paper studies the related problems of prediction, covariance estimation, and principal component analysis for the spiked covariance model with heteroscedastic noise. We consider an estimator of the principal components based on whitening the noise, and we derive optimal singular value and eigenvalue shrinkers for use with these estimated principal components. Underlying these methods are new asymptotic results for the high-dimensional spiked model with heteroscedastic noise, and consistent estimators for the relevant population parameters. We extend previous analysis on out-of-sample prediction to the setting of predictors with whitening. We demonstrate certain advantages of noise whitening. Specifically, we show that in a certain asymptotic regime, optimal singular value shrinkage with whitening converges to the best linear predictor, whereas without whitening it converges to a suboptimal linear predictor. We prove that for generic signals, whitening improves estimation of the principal components, and increases a natural signal-to-noise ratio of the observations. We also show that for rank one signals, our estimated principal components achieve the asymptotic minimax rate.

References in corpus (7)

• Covariance regularization by thresholding
• Single-particle cryo-electron microscopy: Mathematical theory, computational challenges, and opportunities
• Central limit theorems for eigenvalues in a spiked population model
• Denoising and Covariance Estimation of Single Particle Cryo-EM Images
• Asymptotic performance of PCA for high-dimensional heteroscedastic data
• Deterministic parallel analysis: An improved method for selecting factors and principal components
• Towards a Theoretical Analysis of PCA for Heteroscedastic Data

Cited by in corpus (4)

• ScreeNOT: Exact MSE-Optimal Singular Value Thresholding in Correlated Noise
• Entrywise Estimation of Singular Vectors of Low-Rank Matrices with Heteroskedasticity and Dependence
• Biwhitening Reveals the Rank of a Count Matrix
• Matrix Denoising with Partial Noise Statistics: Optimal Singular Value Shrinkage of Spiked F-Matrices