Bayesian Regularization for Graphical Models with Unequal Shrinkage

Abstract

We consider a Bayesian framework for estimating a high-dimensional sparse precision matrix, in which adaptive shrinkage and sparsity are induced by a mixture of Laplace priors. Besides discussing our formulation from the Bayesian standpoint, we investigate the MAP (maximum a posteriori) estimator from a penalized likelihood perspective that gives rise to a new non-convex penalty approximating the ${\ell }_0$ penalty. Optimal error rates for estimation consistency in terms of various matrix norms along with selection consistency for sparse structure recovery are shown for the unique MAP estimator under mild conditions. For fast and efficient computation, an EM algorithm is proposed to compute the MAP estimator of the precision matrix and (approximate) posterior probabilities on the edges of the underlying sparse structure. Through extensive simulation studies and a real application to a call center data, we have demonstrated the fine performance of our method compared with existing alternatives.

arXiv commentTo appear in Journal of the American Statistical Association (Theory & Methods)

References in corpus (6)

• Spike and slab variable selection: Frequentist and Bayesian strategies
• High-dimensional covariance matrix estimation in approximate factor models
• A convex pseudo-likelihood framework for high dimensional partial correlation estimation with convergence guarantees
• Simultaneous Variable and Covariance Selection with the Multivariate Spike-and-Slab Lasso
• Adaptive Lasso for High Dimensional Regression and Gaussian Graphical Modeling
• High-dimensional covariance estimation by minimizing ${\ell }_1$-penalized log-determinant divergence