Sparsistency and rates of convergence in large covariance matrix estimation

Abstract

This paper studies the sparsistency and rates of convergence for estimating sparse covariance and precision matrices based on penalized likelihood with nonconvex penalty functions. Here, sparsistency refers to the property that all parameters that are zero are actually estimated as zero with probability tending to one. Depending on the case of applications, sparsity priori may occur on the covariance matrix, its inverse or its Cholesky decomposition. We study these three sparsity exploration problems under a unified framework with a general penalty function. We show that the rates of convergence for these problems under the Frobenius norm are of order $(s_n\log p_n/n{)}^{1/2}$, where $s_n$ is the number of nonzero elements, $p_n$ is the size of the covariance matrix and $n$ is the sample size. This explicitly spells out the contribution of high-dimensionality is merely of a logarithmic factor. The conditions on the rate with which the tuning parameter ${\lambda }_n$ goes to 0 have been made explicit and compared under different penalties. As a result, for the $L_1$-penalty, to guarantee the sparsistency and optimal rate of convergence, the number of nonzero elements should be small: $s_n^{\prime }=O(p_n)$ at most, among $O(p_n^2)$ parameters, for estimating sparse covariance or correlation matrix, sparse precision or inverse correlation matrix or sparse Cholesky factor, where $s_n^{\prime }$ is the number of the nonzero elements on the off-diagonal entries. On the other hand, using the SCAD or hard-thresholding penalty functions, there is no such a restriction.

Published in at http://dx.doi.org/10.1214/09-AOS720 the Annals of Statistics (http://www.imstat.org/aos/) by the Institute of Mathematical Statistics (http://www.imstat.org)

References in corpus (9)

• High-dimensional graphs and variable selection with the Lasso
• Regularized estimation of large covariance matrices
• Covariance regularization by thresholding
• One-step sparse estimates in nonconcave penalized likelihood models
• Sparse permutation invariant covariance estimation
• Asymptotic properties of bridge estimators in sparse high-dimensional regression models
• Network exploration via the adaptive LASSO and SCAD penalties
• Operator norm consistent estimation of large-dimensional sparse covariance matrices
• Sparse estimation of large covariance matrices via a nested Lasso penalty

Cited by in corpus (52)

• Sparse and compositionally robust inference of microbial ecological networks
• Sparse permutation invariant covariance estimation
• Latent variable graphical model selection via convex optimization
• High-dimensional covariance matrix estimation in approximate factor models
• Strong oracle optimality of folded concave penalized estimation
• Optimal rates of convergence for covariance matrix estimation
• Network exploration via the adaptive LASSO and SCAD penalties
• A Unified Framework for High-Dimensional Analysis of M-Estimators with Decomposable Regularizers
• Asymptotic normality and optimalities in estimation of large Gaussian graphical models
• A sparse conditional Gaussian graphical model for analysis of genetical genomics data
• Regularized rank-based estimation of high-dimensional nonparanormal graphical models
• Covariance Estimation: The GLM and Regularization Perspectives
• Optimal rates of convergence for sparse covariance matrix estimation
• Adaptive covariance matrix estimation through block thresholding
• Covariance and precision matrix estimation for high-dimensional time series
• Fast and Adaptive Sparse Precision Matrix Estimation in High Dimensions
• Gemini: Graph estimation with matrix variate normal instances
• Gaussian and bootstrap approximations for high-dimensional U-statistics and their applications
• Optimal sparse volatility matrix estimation for high-dimensional Itô processes with measurement errors
• Posterior contraction in sparse Bayesian factor models for massive covariance matrices
• Posterior convergence rates for estimating large precision matrices using graphical models
• Asymptotically minimax empirical Bayes estimation of a sparse normal mean vector
• Change-Point Estimation in High-Dimensional Markov Random Field Models
• L0 Sparse Inverse Covariance Estimation
• Graph Estimation From Multi-attribute Data
• Sparsistency and agnostic inference in sparse PCA
• Estimating sufficient reductions of the predictors in abundant high-dimensional regressions
• Robust Sparse Covariance Estimation by Thresholding Tyler's M-Estimator
• Estimation of Toeplitz Covariance Matrices in Large Dimensional Regime with Application to Source Detection
• Estimation of Large Covariance and Precision Matrices from Temporally Dependent Observations
• Fixed support positive-definite modification of covariance matrix estimators via linear shrinkage
• On Sparse High-Dimensional Graphical Model Learning For Dependent Time Series
• Perturbations and projections of Kalman-Bucy semigroups
• Sparse Graph Learning Under Laplacian-Related Constraints
• Optimal covariance matrix estimation for high-dimensional noise in high-frequency data
• Estimation of functionals of sparse covariance matrices
• Shrinking characteristics of precision matrix estimators
• A review of Gaussian Markov models for conditional independence
• Certifiably Optimal Sparse Inverse Covariance Estimation
• De-biased graphical Lasso for high-frequency data
• Random Matrix Derived Shrinkage of Spectral Precision Matrices
• Cross-Validated Loss-Based Covariance Matrix Estimator Selection in High Dimensions
• A Generative Approach to Joint Modeling of Quantitative and Qualitative Responses
• Penalized Sparse Covariance Regression with High Dimensional Covariates
• Graph Coding for Model Selection and Anomaly Detection in Gaussian Graphical Models
• A random covariance model for bi-level graphical modeling with application to resting-state fMRI data
• An efficient numerical method for condition number constrained covariance matrix approximation
• Learning Multi-Attribute Differential Graphs with Non-Convex Penalties
• On Conditional Independence Graph Learning From Multi-Attribute Gaussian Dependent Time Series
• Learning Graph Laplacian with MCP
• CDPA: Common and Distinctive Pattern Analysis between High-dimensional Datasets
• Multi-Attribute Graph Estimation with Sparse-Group Non-Convex Penalties