Sparse permutation invariant covariance estimation

Abstract

The paper proposes a method for constructing a sparse estimator for the inverse covariance (concentration) matrix in high-dimensional settings. The estimator uses a penalized normal likelihood approach and forces sparsity by using a lasso-type penalty. We establish a rate of convergence in the Frobenius norm as both data dimension $p$ and sample size $n$ are allowed to grow, and show that the rate depends explicitly on how sparse the true concentration matrix is. We also show that a correlation-based version of the method exhibits better rates in the operator norm. We also derive a fast iterative algorithm for computing the estimator, which relies on the popular Cholesky decomposition of the inverse but produces a permutation-invariant estimator. The method is compared to other estimators on simulated data and on a real data example of tumor tissue classification using gene expression data.

Published in at http://dx.doi.org/10.1214/08-EJS176 the Electronic Journal of Statistics (http://www.i-journals.org/ejs/) by the Institute of Mathematical Statistics (http://www.imstat.org)

References in corpus (10)

• High-dimensional graphs and variable selection with the Lasso
• Pathwise coordinate optimization
• Regularized estimation of large covariance matrices
• Covariance regularization by thresholding
• Estimating high-dimensional directed acyclic graphs with the PC-algorithm
• Sparsistency and rates of convergence in large covariance matrix estimation
• Variable selection using MM algorithms
• Operator norm consistent estimation of large-dimensional sparse covariance matrices
• Sparse estimation of large covariance matrices via a nested Lasso penalty
• Estimation of a Covariance Matrix with Zeros

Cited by in corpus (78)

• Covariance regularization by thresholding
• High-dimensional regression with noisy and missing data: Provable guarantees with nonconvexity
• High-dimensional Ising model selection using ${\ell }_1$-regularized logistic regression
• Sparsistency and rates of convergence in large covariance matrix estimation
• Latent variable graphical model selection via convex optimization
• Optimal rates of convergence for covariance matrix estimation
• Network exploration via the adaptive LASSO and SCAD penalties
• A Unified Framework for High-Dimensional Analysis of M-Estimators with Decomposable Regularizers
• Regularized estimation of large-scale gene association networks using graphical Gaussian models
• Asymptotic normality and optimalities in estimation of large Gaussian graphical models
• Regularized rank-based estimation of high-dimensional nonparanormal graphical models
• Covariance Estimation: The GLM and Regularization Perspectives
• Estimating time-varying networks
• Adaptive covariance matrix estimation through block thresholding
• Confidence intervals for high-dimensional inverse covariance estimation
• High-dimensionality effects in the Markowitz problem and other quadratic programs with linear constraints: Risk underestimation
• Covariance Estimation in High Dimensions via Kronecker Product Expansions
• Covariance and precision matrix estimation for high-dimensional time series
• Structure estimation for discrete graphical models: Generalized covariance matrices and their inverses
• Missing values: sparse inverse covariance estimation and an extension to sparse regression
• Generalized Symmetric ADMM for Separable Convex Optimization
• Optimal computational and statistical rates of convergence for sparse nonconvex learning problems
• Scaling It Up: Stochastic Search Structure Learning in Graphical Models
• Transposable regularized covariance models with an application to missing data imputation
• Fast and Adaptive Sparse Precision Matrix Estimation in High Dimensions
• Convergence Properties of Kronecker Graphical Lasso Algorithms
• Gemini: Graph estimation with matrix variate normal instances
• Gaussian and bootstrap approximations for high-dimensional U-statistics and their applications
• Statistical Significance of Clustering using Soft Thresholding
• Posterior convergence rates for estimating large precision matrices using graphical models
• Innovated interaction screening for high-dimensional nonlinear classification
• Learning Gaussian Tree Models: Analysis of Error Exponents and Extremal Structures
• Tyler's Covariance Matrix Estimator in Elliptical Models with Convex Structure
• L0 Sparse Inverse Covariance Estimation
• Bootstrap inference for network construction with an application to a breast cancer microarray study
• Performance Analysis of Tyler's Covariance Estimator
• Markov models for fMRI correlation structure: is brain functional connectivity small world, or decomposable into networks?
• Estimating heterogeneous graphical models for discrete data with an application to roll call voting
• Sparsistency and agnostic inference in sparse PCA
• Estimating sufficient reductions of the predictors in abundant high-dimensional regressions
• Sparse Nonparametric Graphical Models
• Improved multivariate normal mean estimation with unknown covariance when p is greater than n
• Penalized estimation in high-dimensional hidden Markov models with state-specific graphical models
• Marginal Likelihoods for Distributed Parameter Estimation of Gaussian Graphical Models
• A Proximal Approach for a Class of Matrix Optimization Problems
• Estimation of Large Covariance and Precision Matrices from Temporally Dependent Observations
• Linear Estimating Equations for Exponential Families with Application to Gaussian Linear Concentration Models
• High dimensional sparse covariance estimation via directed acyclic graphs
• On Sparse High-Dimensional Graphical Model Learning For Dependent Time Series
• Statistical Perspective on Functional and Causal Neural Connectomics: A Comparative Study
• Sparse Graph Learning Under Laplacian-Related Constraints
• L1-Penalized Censored Gaussian Graphical Model
• Bayesian Inference in Nonparanormal Graphical Models
• Network reconstruction via the minimum description length principle
• Shrinking characteristics of precision matrix estimators
• Discussion: Latent variable graphical model selection via convex optimization
• Certifiably Optimal Sparse Inverse Covariance Estimation
• A review of Gaussian Markov models for conditional independence
• Online Structural Change-point Detection of High-dimensional Streaming Data via Dynamic Sparse Subspace Learning
• Efficient Distributed Estimation of Inverse Covariance Matrices
• A Proximal Point Dual Newton Algorithm for Solving Group Graphical Lasso Problems
• Random Matrix Derived Shrinkage of Spectral Precision Matrices
• De-biased graphical Lasso for high-frequency data
• Estimation of dynamic networks for high-dimensional nonstationary time series
• A Generative Approach to Joint Modeling of Quantitative and Qualitative Responses
• An efficiency upper bound for inverse covariance estimation
• Discussion: Latent variable graphical model selection via convex optimization
• Discussion: Latent variable graphical model selection via convex optimization
• A random covariance model for bi-level graphical modeling with application to resting-state fMRI data
• Sparse Cholesky covariance parametrization for recovering latent structure in ordered data
• Graph Coding for Model Selection and Anomaly Detection in Gaussian Graphical Models
• A Mixture Model to Detect Edges in Sparse Co-expression Graphs
• On Conditional Independence Graph Learning From Multi-Attribute Gaussian Dependent Time Series
• A Bayesian Graphical Approach for Large-Scale Portfolio Management with Fewer Historical Data
• Learning Graph Laplacian with MCP
• Multi-Attribute Graph Estimation with Sparse-Group Non-Convex Penalties
• Efficient proximal gradient algorithms for joint graphical lasso
• Thinning a Wishart Random Matrix