Posterior contraction in sparse Bayesian factor models for massive covariance matrices

Abstract

Sparse Bayesian factor models are routinely implemented for parsimonious dependence modeling and dimensionality reduction in high-dimensional applications. We provide theoretical understanding of such Bayesian procedures in terms of posterior convergence rates in inferring high-dimensional covariance matrices where the dimension can be larger than the sample size. Under relevant sparsity assumptions on the true covariance matrix, we show that commonly-used point mass mixture priors on the factor loadings lead to consistent estimation in the operator norm even when $p\gg n$. One of our major contributions is to develop a new class of continuous shrinkage priors and provide insights into their concentration around sparse vectors. Using such priors for the factor loadings, we obtain similar rate of convergence as obtained with point mass mixture priors. To obtain the convergence rates, we construct test functions to separate points in the space of high-dimensional covariance matrices using insights from random matrix theory; the tools developed may be of independent interest. We also derive minimax rates and show that the Bayesian posterior rates of convergence coincide with the minimax rates upto a $\sqrt{\log n}$ term.

Published in at http://dx.doi.org/10.1214/14-AOS1215 the Annals of Statistics (http://www.imstat.org/aos/) by the Institute of Mathematical Statistics (http://www.imstat.org)

References in corpus (13)

• Regularized estimation of large covariance matrices
• Covariance regularization by thresholding
• Bayes and empirical-Bayes multiplicity adjustment in the variable-selection problem
• Optimal rates of convergence for covariance matrix estimation
• Operator norm consistent estimation of large-dimensional sparse covariance matrices
• Needles and Straw in a Haystack: Posterior concentration for possibly sparse sequences
• Convergence rates of posterior distributions for noniid observations
• Optimal rates of convergence for sparse covariance matrix estimation
• Factor modeling for high-dimensional time series: Inference for the number of factors
• Bayesian variable selection for high dimensional generalized linear models: convergence rates of the fitted densities
• On the sample covariance matrix estimator of reduced effective rank population matrices, with applications to fPCA
• Bayesian inverse problems with non-conjugate priors
• Rates of contraction for posterior distributions in $\bolds{L^r}$-metrics, $\bolds{1\le r\le\infty}$

Cited by in corpus (8)

• Sparse Bayesian time-varying covariance estimation in many dimensions
• Sparse Bayesian vector autoregressions in huge dimensions
• Posterior convergence rates for estimating large precision matrices using graphical models
• Asymptotically minimax empirical Bayes estimation of a sparse normal mean vector
• Rate-optimal posterior contraction for sparse PCA
• Bayesian joint modeling of chemical structure and dose response curves
• Adaptive variational Bayes: Optimality, computation and applications
• A note on simulation methods for the Dirichlet-Laplace prior