Regularized estimation of large covariance matrices

Abstract

This paper considers estimating a covariance matrix of $p$ variables from $n$ observations by either banding or tapering the sample covariance matrix, or estimating a banded version of the inverse of the covariance. We show that these estimates are consistent in the operator norm as long as $(\log p)/n\to 0$, and obtain explicit rates. The results are uniform over some fairly natural well-conditioned families of covariance matrices. We also introduce an analogue of the Gaussian white noise model and show that if the population covariance is embeddable in that model and well-conditioned, then the banded approximations produce consistent estimates of the eigenvalues and associated eigenvectors of the covariance matrix. The results can be extended to smooth versions of banding and to non-Gaussian distributions with sufficiently short tails. A resampling approach is proposed for choosing the banding parameter in practice. This approach is illustrated numerically on both simulated and real data.

Published in at http://dx.doi.org/10.1214/009053607000000758 the Annals of Statistics (http://www.imstat.org/aos/) by the Institute of Mathematical Statistics (http://www.imstat.org)

Cited by in corpus (10)

• Covariance regularization by thresholding
• Sparse permutation invariant covariance estimation
• Non-Euclidean statistics for covariance matrices, with applications to diffusion tensor imaging
• Operator norm consistent estimation of large-dimensional sparse covariance matrices
• Sparse estimation of large covariance matrices via a nested Lasso penalty
• High-dimensional analysis of semidefinite relaxations for sparse principal components
• Flexible covariance estimation in graphical Gaussian models
• Treelets--An adaptive multi-scale basis for sparse unordered data
• Semiparametric detection of significant activation for brain fMRI
• A CLT for regularized sample covariance matrices