L0 Sparse Inverse Covariance Estimation

Abstract

Recently, there has been focus on penalized log-likelihood covariance estimation for sparse inverse covariance (precision) matrices. The penalty is responsible for inducing sparsity, and a very common choice is the convex $l_1$ norm. However, the best estimator performance is not always achieved with this penalty. The most natural sparsity promoting "norm" is the non-convex $l_0$ penalty but its lack of convexity has deterred its use in sparse maximum likelihood estimation. In this paper we consider non-convex $l_0$ penalized log-likelihood inverse covariance estimation and present a novel cyclic descent algorithm for its optimization. Convergence to a local minimizer is proved, which is highly non-trivial, and we demonstrate via simulations the reduced bias and superior quality of the $l_0$ penalty as compared to the $l_1$ penalty.

References in corpus (4)

• Extended Bayesian Information Criteria for Gaussian Graphical Models
• Sparse permutation invariant covariance estimation
• Network exploration via the adaptive LASSO and SCAD penalties
• Projected Subgradient Methods for Learning Sparse Gaussians

Cited by in corpus (10)

• Regularized estimation of linear functionals of precision matrices for high-dimensional time series
• Certifiably Optimal Sparse Inverse Covariance Estimation
• Learning Directed Graphical Models from Gaussian Data
• Testing and Support Recovery of Correlation Structures for Matrix-Valued Observations with an Application to Stock Market Data
• A Coordinate-wise Optimization Algorithm for Sparse Inverse Covariance Selection
• A Survey on Nonconvex Regularization Based Sparse and Low-Rank Recovery in Signal Processing, Statistics, and Machine Learning
• Sparse sketches with small inversion bias
• Combining Smoothing Spline with Conditional Gaussian Graphical Model for Density and Graph Estimation
• Semiblind subgraph reconstruction in Gaussian graphical models
• High-dimensional Simultaneous Inference on Non-Gaussian VAR Model via De-biased Estimator