Covariance regularization by thresholding

Abstract

This paper considers regularizing a covariance matrix of $p$ variables estimated from $n$ observations, by hard thresholding. We show that the thresholded estimate is consistent in the operator norm as long as the true covariance matrix is sparse in a suitable sense, the variables are Gaussian or sub-Gaussian, and $(\log p)/n\to 0$, and obtain explicit rates. The results are uniform over families of covariance matrices which satisfy a fairly natural notion of sparsity. We discuss an intuitive resampling scheme for threshold selection and prove a general cross-validation result that justifies this approach. We also compare thresholding to other covariance estimators in simulations and on an example from climate data.

Published in at http://dx.doi.org/10.1214/08-AOS600 the Annals of Statistics (http://www.imstat.org/aos/) by the Institute of Mathematical Statistics (http://www.imstat.org)

References in corpus (8)

• Regularized estimation of large covariance matrices
• Sparse permutation invariant covariance estimation
• Empirical Bayes selection of wavelet thresholds
• Network exploration via the adaptive LASSO and SCAD penalties
• Operator norm consistent estimation of large-dimensional sparse covariance matrices
• Tracy--Widom limit for the largest eigenvalue of a large class of complex sample covariance matrices
• Sparse Principal Components Analysis
• Sparse estimation of large covariance matrices via a nested Lasso penalty

Cited by in corpus (107)

• Challenges of Big Data Analysis
• Sparse permutation invariant covariance estimation
• Sparsistency and rates of convergence in large covariance matrix estimation
• Latent variable graphical model selection via convex optimization
• High-dimensional covariance matrix estimation in approximate factor models
• Regularized estimation in sparse high-dimensional time series models
• Optimal rates of convergence for covariance matrix estimation
• Sparse PCA: Optimal rates and adaptive estimation
• A Unified Framework for High-Dimensional Analysis of M-Estimators with Decomposable Regularizers
• Optimal detection of sparse principal components in high dimension
• Geometric median and robust estimation in Banach spaces
• Operator norm consistent estimation of large-dimensional sparse covariance matrices
• Two sample tests for high-dimensional covariance matrices
• Asymptotic normality and optimalities in estimation of large Gaussian graphical models
• Covariance Estimation: The GLM and Regularization Perspectives
• Sparse linear discriminant analysis by thresholding for high dimensional data
• Projected principal component analysis in factor models
• Optimal rates of convergence for sparse covariance matrix estimation
• Vast volatility matrix estimation for high-frequency financial data
• Coauthorship and Citation Networks for Statisticians
• Adaptive covariance matrix estimation through block thresholding
• Confidence intervals for high-dimensional inverse covariance estimation
• Covariance matrix estimation for stationary time series
• High-dimensionality effects in the Markowitz problem and other quadratic programs with linear constraints: Risk underestimation
• Covariance and precision matrix estimation for high-dimensional time series
• Constrained Factor Models for High-Dimensional Matrix-Variate Time Series
• High-dimensional variable selection via tilting
• Scaling It Up: Stochastic Search Structure Learning in Graphical Models
• Multilevel ensemble Kalman filtering
• Fast and Adaptive Sparse Precision Matrix Estimation in High Dimensions
• Bayesian Sparse Factor Analysis of Genetic Covariance Matrices
• Phylogenetic analysis of gene expression
• Asymptotic performance of PCA for high-dimensional heteroscedastic data
• Gaussian and bootstrap approximations for high-dimensional U-statistics and their applications
• Test for bandedness of high-dimensional covariance matrices and bandwidth estimation
• Optimal sparse volatility matrix estimation for high-dimensional Itô processes with measurement errors
• Posterior contraction in sparse Bayesian factor models for massive covariance matrices
• Moving Beyond Sub-Gaussianity in High-Dimensional Statistics: Applications in Covariance Estimation and Linear Regression
• Posterior convergence rates for estimating large precision matrices using graphical models
• Networks, Dynamic Factors, and the Volatility Analysis of High-Dimensional Financial Series
• On the sample covariance matrix estimator of reduced effective rank population matrices, with applications to fPCA
• Principal component analysis for second-order stationary vector time series
• UPS delivers optimal phase diagram in high-dimensional variable selection
• Statistical analysis of latent generalized correlation matrix estimation in transelliptical distribution
• Covariance estimation in decomposable Gaussian graphical models
• Regression on manifolds: Estimation of the exterior derivative
• Modelling matrix time series via a tensor CP-decomposition
• Confidence regions for entries of a large precision matrix
• A penalized empirical likelihood method in high dimensions
• Finite sample change point inference and identification for high-dimensional mean vectors
• Linear Shrinkage Estimation of Covariance Matrices Using Low-Complexity Cross-Validation
• Detection of correlations
• Adaptive estimation of the copula correlation matrix for semiparametric elliptical copulas
• Complete characterization of Hadamard powers preserving Loewner positivity, monotonicity, and convexity
• High-dimensional influence measure
• Estimating sufficient reductions of the predictors in abundant high-dimensional regressions
• Robust Sparse Covariance Estimation by Thresholding Tyler's M-Estimator
• ScreeNOT: Exact MSE-Optimal Singular Value Thresholding in Correlated Noise
• Regularized estimation of linear functionals of precision matrices for high-dimensional time series
• Introduction to correlation networks: Interdisciplinary approaches beyond thresholding
• Cholesky-based multivariate Gaussian regression
• Preserving positivity for rank-constrained matrices
• Estimation of Large Covariance and Precision Matrices from Temporally Dependent Observations
• Exact Blind Community Detection from Signals on Multiple Graphs
• High dimensional sparse covariance estimation via directed acyclic graphs
• Matrix positivity preservers in fixed dimension. I
• Detecting positive correlations in a multivariate sample
• Critical exponents of graphs
• Fixed support positive-definite modification of covariance matrix estimators via linear shrinkage
• Next Generation Models for Portfolio Risk Management: An Approach Using Financial Big Data
• Optimal spectral shrinkage and PCA with heteroscedastic noise
• Multi-Scale Factor Analysis of High-Dimensional Brain Signals
• Scattering Spectra Models for Physics
• Structural shrinkage of nonparametric spectral estimators for multivariate time series
• Non-Asymptotic Analysis of Ensemble Kalman Updates: Effective Dimension and Localization
• Optimal covariance matrix estimation for high-dimensional noise in high-frequency data
• Large-sample approximations for variance-covariance matrices of high-dimensional time series
• A large covariance matrix estimator under intermediate spikiness regimes
• Estimation of functionals of sparse covariance matrices
• Certifiably Optimal Sparse Inverse Covariance Estimation
• SURE Information Criteria for Large Covariance Matrix Estimation and Their Asymptotic Properties
• De-biased graphical Lasso for high-frequency data
• Implied Basket Correlation Dynamics
• Covariance Matrix Estimation from Linearly-Correlated Gaussian Samples
• Cross-Validated Loss-Based Covariance Matrix Estimator Selection in High Dimensions
• Covariance Matrix Estimation for High-Throughput Biomedical Data with Interconnected Communities
• R-NL: Covariance Matrix Estimation for Elliptical Distributions based on Nonlinear Shrinkage
• Penalized Sparse Covariance Regression with High Dimensional Covariates
• A Generative Approach to Joint Modeling of Quantitative and Qualitative Responses
• Irrational Exuberance: Correcting Bias in Probability Estimates
• Skilled Mutual Fund Selection: False Discovery Control under Dependence
• Estimating covariance and precision matrices along subspaces
• Estimation of the Kronecker Covariance Model by Quadratic Form
• Regularizing stock return covariance matrices via multiple testing of correlations
• Efficient Computation of Sparse and Robust Maximum Association Estimators
• Nonasymptotic estimation and support recovery for high dimensional sparse covariance matrices
• An efficient numerical method for condition number constrained covariance matrix approximation
• Detecting the large entries of a sparse covariance matrix in sub-quadratic time
• Tyler's and Maronna's M-estimators: Non-Asymptotic Concentration Results
• Covariance Operator Estimation via Adaptive Thresholding
• Nonparametric Estimation of Large Spot Volatility Matrices for High-Frequency Financial Data
• Cross-validated covariance estimators for high-dimensional minimum-variance portfolios
• Hub Detection in Gaussian Graphical Models
• Simultaneous Estimation of Multiple Treatment Effects from Observational Studies
• Some Statistical Problems with High Dimensional Financial data
• Large Global Volatility Matrix Analysis Based on Observation Structural Information
• The Generalized Elastic Net for least squares regression with network-aligned signal and correlated design