Regularized estimation of large covariance matrices

Abstract

This paper considers estimating a covariance matrix of $p$ variables from $n$ observations by either banding or tapering the sample covariance matrix, or estimating a banded version of the inverse of the covariance. We show that these estimates are consistent in the operator norm as long as $(\log p)/n\to 0$, and obtain explicit rates. The results are uniform over some fairly natural well-conditioned families of covariance matrices. We also introduce an analogue of the Gaussian white noise model and show that if the population covariance is embeddable in that model and well-conditioned, then the banded approximations produce consistent estimates of the eigenvalues and associated eigenvectors of the covariance matrix. The results can be extended to smooth versions of banding and to non-Gaussian distributions with sufficiently short tails. A resampling approach is proposed for choosing the banding parameter in practice. This approach is illustrated numerically on both simulated and real data.

Published in at http://dx.doi.org/10.1214/009053607000000758 the Annals of Statistics (http://www.imstat.org/aos/) by the Institute of Mathematical Statistics (http://www.imstat.org)

References in corpus (2)

• High-dimensional graphs and variable selection with the Lasso
• Sparse principal component analysis and iterative thresholding

Cited by in corpus (125)

• Covariance regularization by thresholding
• Shrinkage Algorithms for MMSE Covariance Estimation
• Sparse permutation invariant covariance estimation
• Nonlinear shrinkage estimation of large-dimensional covariance matrices
• Sparsistency and rates of convergence in large covariance matrix estimation
• Non-Euclidean statistics for covariance matrices, with applications to diffusion tensor imaging
• Latent variable graphical model selection via convex optimization
• High-dimensional covariance matrix estimation in approximate factor models
• Estimation of high-dimensional low-rank matrices
• Strong oracle optimality of folded concave penalized estimation
• Optimal rates of convergence for covariance matrix estimation
• Robust Shrinkage Estimation of High-dimensional Covariance Matrices
• Geometric median and robust estimation in Banach spaces
• Operator norm consistent estimation of large-dimensional sparse covariance matrices
• Two sample tests for high-dimensional covariance matrices
• Asymptotic normality and optimalities in estimation of large Gaussian graphical models
• Covariance Estimation: The GLM and Regularization Perspectives
• Computational and Statistical Tradeoffs via Convex Relaxation
• Innovated higher criticism for detecting sparse signals in correlated noise
• Sparse estimation of large covariance matrices via a nested Lasso penalty
• Optimal rates of convergence for sparse covariance matrix estimation
• Vast volatility matrix estimation for high-frequency financial data
• Coauthorship and Citation Networks for Statisticians
• Higher Criticism for Large-Scale Inference, Especially for Rare and Weak Effects
• Adaptive covariance matrix estimation through block thresholding
• Covariance matrix estimation for stationary time series
• High-dimensional analysis of semidefinite relaxations for sparse principal components
• High-dimensionality effects in the Markowitz problem and other quadratic programs with linear constraints: Risk underestimation
• Robust Estimation of Structured Covariance Matrix for Heavy-Tailed Elliptical Distributions
• Covariance Estimation in High Dimensions via Kronecker Product Expansions
• The proximal augmented Lagrangian method for nonsmooth composite optimization
• Multivariate Generalized Gaussian Distribution: Convexity and Graphical Models
• Concentration of measure and spectra of random matrices: Applications to correlation matrices, elliptical distributions and beyond
• Inference of time-varying regression models
• Covariance and precision matrix estimation for high-dimensional time series
• Multilevel ensemble Kalman filtering
• Bayesian Sparse Factor Analysis of Genetic Covariance Matrices
• Flexible covariance estimation in graphical Gaussian models
• Persistent Homology in Sparse Regression and its Application to Brain Morphometry
• Treelets--An adaptive multi-scale basis for sparse unordered data
• Gaussian and bootstrap approximations for high-dimensional U-statistics and their applications
• Test for bandedness of high-dimensional covariance matrices and bandwidth estimation
• Posterior contraction in sparse Bayesian factor models for massive covariance matrices
• Optimal sparse volatility matrix estimation for high-dimensional Itô processes with measurement errors
• Posterior convergence rates for estimating large precision matrices using graphical models
• Innovated interaction screening for high-dimensional nonlinear classification
• Factor models and variable selection in high-dimensional regression analysis
• On the sample covariance matrix estimator of reduced effective rank population matrices, with applications to fPCA
• Mean--variance portfolio optimization when means and covariances are unknown
• Tyler's Covariance Matrix Estimator in Elliptical Models with Convex Structure
• Orthogonal Sparse PCA and Covariance Estimation via Procrustes Reformulation
• Statistical analysis of latent generalized correlation matrix estimation in transelliptical distribution
• Independence test for high dimensional data based on regularized canonical correlation coefficients
• Testing for Independence of Large Dimensional Vectors
• Covariance estimation in decomposable Gaussian graphical models
• A whitening approach to probabilistic canonical correlation analysis for omics data integration
• Confidence regions for entries of a large precision matrix
• Statistical Inference in Large Antenna Arrays under Unknown Noise Pattern
• Stable Estimation of a Covariance Matrix Guided by Nuclear Norm Penalties
• Finite sample change point inference and identification for high-dimensional mean vectors
• Performance Analysis of Tyler's Covariance Estimator
• Linear Shrinkage Estimation of Covariance Matrices Using Low-Complexity Cross-Validation
• Do semidefinite relaxations solve sparse PCA up to the information limit?
• Markov models for fMRI correlation structure: is brain functional connectivity small world, or decomposable into networks?
• Estimating sufficient reductions of the predictors in abundant high-dimensional regressions
• Discussion: Latent variable graphical model selection via convex optimization
• Optimal classification in sparse Gaussian graphic model
• Improved multivariate normal mean estimation with unknown covariance when p is greater than n
• Bayesian sparse graphical models for classification with application to protein expression data
• Estimation of Toeplitz Covariance Matrices in Large Dimensional Regime with Application to Source Detection
• Regularized estimation of linear functionals of precision matrices for high-dimensional time series
• Estimation of Large Covariance and Precision Matrices from Temporally Dependent Observations
• Linear Estimating Equations for Exponential Families with Application to Gaussian Linear Concentration Models
• High dimensional sparse covariance estimation via directed acyclic graphs
• Large Dimensional Analysis of Robust M-Estimators of Covariance with Outliers
• Semiparametric detection of significant activation for brain fMRI
• Regularized Covariance Estimation for Polarization Radar Detection in Compound Gaussian Sea Clutter
• A Bayesian Approach to Multivariate Adaptive Localization in Ensemble-Based Data Assimilation with Time-Dependent Extensions
• Performance analysis of local ensemble Kalman filter
• Detecting positive correlations in a multivariate sample
• Fixed support positive-definite modification of covariance matrix estimators via linear shrinkage
• Improving compressed sensing with the diamond norm
• Perturbations and projections of Kalman-Bucy semigroups
• Analysis of a localised nonlinear Ensemble Kalman Bucy Filter with complete and accurate observations
• Non-Asymptotic Analysis of Ensemble Kalman Updates: Effective Dimension and Localization
• Optimal covariance matrix estimation for high-dimensional noise in high-frequency data
• A large covariance matrix estimator under intermediate spikiness regimes
• L1-Penalized Censored Gaussian Graphical Model
• An approach to the distributionally robust shortest path problem
• Tests for covariance matrix with fixed or divergent dimension
• A Flexible Approach for Predictive Biomarker Discovery
• Estimation of functionals of sparse covariance matrices
• A CLT for regularized sample covariance matrices
• Optimal properties of centroid-based classifiers for very high-dimensional data
• Shrinking characteristics of precision matrix estimators
• Optimal Discrete Spatial Compression for Beamspace Massive MIMO Signals
• Refining genetically inferred relationships using treelet covariance smoothing
• Joint Association Graph Screening and Decomposition for Large-scale Linear Dynamical Systems
• SURE Information Criteria for Large Covariance Matrix Estimation and Their Asymptotic Properties
• Implied Basket Correlation Dynamics
• De-biased graphical Lasso for high-frequency data
• Covariance Matrix Estimation for High-Throughput Biomedical Data with Interconnected Communities
• Cross-Validated Loss-Based Covariance Matrix Estimator Selection in High Dimensions
• Covariance Matrix Estimation from Linearly-Correlated Gaussian Samples
• Panel data segmentation under finite time horizon
• R-NL: Covariance Matrix Estimation for Elliptical Distributions based on Nonlinear Shrinkage
• Estimation of inverse autocovariance matrices for long memory processes
• Penalized Sparse Covariance Regression with High Dimensional Covariates
• Asymptotic distribution of least square estimators for linear models with dependent errors
• Random Matrix Improved Covariance Estimation for a Large Class of Metrics
• Towards the interpretation of time-varying regularization parameters in streaming penalized regression models
• An Asymptotic Analysis of Minibatch-Based Momentum Methods for Linear Regression Models
• A random covariance model for bi-level graphical modeling with application to resting-state fMRI data
• Estimating covariance and precision matrices along subspaces
• Regularizing stock return covariance matrices via multiple testing of correlations
• Principal Components of Short-term Variability in Venus' UV Albedo
• Joint Covariance Estimation with Mutual Linear Structure
• An Orthogonally Equivariant Estimator of the Covariance Matrix in High Dimensions and for Small Sample Sizes
• Nonasymptotic estimation and support recovery for high dimensional sparse covariance matrices
• A Structured Approach to the Analysis of Remote Sensing Images
• Eigengap Sparsity for Covariance Parsimony
• Covariance Operator Estimation via Adaptive Thresholding
• Posterior consistency in multi-response regression models with non-informative priors for the error covariance matrix in growing dimensions
• Hub Detection in Gaussian Graphical Models
• Geometric medians on product manifolds