Iterative Thresholding Algorithm for Sparse Inverse Covariance Estimation
arXiv:1211.2532
Abstract
The L1-regularized maximum likelihood estimation problem has recently become a topic of great interest within the machine learning, statistics, and optimization communities as a method for producing sparse inverse covariance estimators. In this paper, a proximal gradient method (G-ISTA) for performing L1-regularized covariance matrix estimation is presented. Although numerous algorithms have been proposed for solving this problem, this simple proximal gradient method is found to have attractive theoretical and numerical properties. G-ISTA has a linear rate of convergence, resulting in an O(log e) iteration complexity to reach a tolerance of e. This paper gives eigenvalue bounds for the G-ISTA iterates, providing a closed-form linear convergence rate. The rate is shown to be closely related to the condition number of the optimal point. Numerical convergence results and timing comparisons for the proposed method are presented. G-ISTA is shown to perform very well, especially when the optimal point is well-conditioned.
25 pages, 1 figure, 4 tables. Conference paper
Cited by in corpus (7)
- Embarrassingly Shallow Autoencoders for Sparse Data
- A proximal Newton framework for composite minimization: Graph learning without Cholesky decompositions and matrix inversions
- G-AMA: Sparse Gaussian graphical model estimation via alternating minimization
- A Scalable Algorithm for Gaussian Graphical Models with Change-Points
- Structure Learning of Gaussian Markov Random Fields with False Discovery Rate Control
- Exact Hybrid Covariance Thresholding for Joint Graphical Lasso
- Estimating Multiple Precision Matrices with Cluster Fusion Regularization