De-biased sparse PCA: Inference and testing for eigenstructure of large covariance matrices

Abstract

Sparse principal component analysis (sPCA) has become one of the most widely used techniques for dimensionality reduction in high-dimensional datasets. The main challenge underlying sPCA is to estimate the first vector of loadings of the population covariance matrix, provided that only a certain number of loadings are non-zero. In this paper, we propose confidence intervals for individual loadings and for the largest eigenvalue of the population covariance matrix. Given an independent sample $X^i\in {\mathbb{R}}^p,i=1,...,n,$ generated from an unknown distribution with an unknown covariance matrix ${\Sigma }_0$, our aim is to estimate the first vector of loadings and the largest eigenvalue of ${\Sigma }_0$ in a setting where $p\gg n$. Next to the high-dimensionality, another challenge lies in the inherent non-convexity of the problem. We base our methodology on a Lasso-penalized M-estimator which, despite non-convexity, may be solved by a polynomial-time algorithm such as coordinate or gradient descent. We show that our estimator achieves the minimax optimal rates in ${\ell }_1$ and ${\ell }_2$-norm. We identify the bias in the Lasso-based estimator and propose a de-biased sparse PCA estimator for the vector of loadings and for the largest eigenvalue of the covariance matrix ${\Sigma }_0$. Our main results provide theoretical guarantees for asymptotic normality of the de-biased estimator. The major conditions we impose are sparsity in the first eigenvector of small order $\sqrt{n}/\log p$ and sparsity of the same order in the columns of the inverse Hessian matrix of the population risk.

41 pages

References in corpus (9)

• Confidence Intervals and Hypothesis Testing for High-Dimensional Regression
• High-dimensional regression with noisy and missing data: Provable guarantees with nonconvexity
• Sparse PCA: Optimal rates and adaptive estimation
• Optimal detection of sparse principal components in high dimension
• Uniform Post Selection Inference for LAD Regression and Other Z-estimation problems
• Normal approximation and concentration of spectral projectors of sample covariance
• Sparse PCA via Covariance Thresholding
• Asymptotics of Empirical Eigen-structure for Ultra-high Dimensional Spiked Covariance Model
• Surprising Asymptotic Conical Structure in Critical Sample Eigen-Directions

Cited by in corpus (5)

• Inference and Uncertainty Quantification for Noisy Matrix Completion
• Singular vector and singular subspace distribution for the matrix denoising model
• Tackling small eigen-gaps: Fine-grained eigenvector estimation and inference under heteroscedastic noise
• Sparse spectral estimation with missing and corrupted measurements
• Estimating high-dimensional Markov-switching VARs