Rate-optimal posterior contraction for sparse PCA

Abstract

Principal component analysis (PCA) is possibly one of the most widely used statistical tools to recover a low-rank structure of the data. In the high-dimensional settings, the leading eigenvector of the sample covariance can be nearly orthogonal to the true eigenvector. A sparse structure is then commonly assumed along with a low rank structure. Recently, minimax estimation rates of sparse PCA were established under various interesting settings. On the other side, Bayesian methods are becoming more and more popular in high-dimensional estimation, but there is little work to connect frequentist properties and Bayesian methodologies for high-dimensional data analysis. In this paper, we propose a prior for the sparse PCA problem and analyze its theoretical properties. The prior adapts to both sparsity and rank. The posterior distribution is shown to contract to the truth at optimal minimax rates. In addition, a computationally efficient strategy for the rank-one case is discussed.

Published at http://dx.doi.org/10.1214/14-AOS1268 in the Annals of Statistics (http://www.imstat.org/aos/) by the Institute of Mathematical Statistics (http://www.imstat.org)

References in corpus (6)

• Rates of contraction of posterior distributions based on Gaussian process priors
• Needles and Straw in a Haystack: Posterior concentration for possibly sparse sequences
• High-dimensional analysis of semidefinite relaxations for sparse principal components
• The Horseshoe Estimator: Posterior Concentration around Nearly Black Vectors
• Augmented sparse principal component analysis for high dimensional data
• Lower bounds for posterior rates with Gaussian process priors

Cited by in corpus (2)

• Minimax estimation in sparse canonical correlation analysis
• Bayesian inference for spectral projectors of the covariance matrix