A majorization-minimization approach to variable selection using spike and slab priors

Abstract

We develop a method to carry out MAP estimation for a class of Bayesian regression models in which coefficients are assigned with Gaussian-based spike and slab priors. The objective function in the corresponding optimization problem has a Lagrangian form in that regression coefficients are regularized by a mixture of squared $l_2$ and $l_0$ norms. A tight approximation to the $l_0$ norm using majorization-minimization techniques is derived, and a coordinate descent algorithm in conjunction with a soft-thresholding scheme is used in searching for the optimizer of the approximate objective. Simulation studies show that the proposed method can lead to more accurate variable selection than other benchmark methods. Theoretical results show that under regular conditions, sign consistency can be established, even when the Irrepresentable Condition is violated. Results on posterior model consistency and estimation consistency, and an extension to parameter estimation in the generalized linear models are provided.

Published in at http://dx.doi.org/10.1214/11-AOS884 the Annals of Statistics (http://www.imstat.org/aos/) by the Institute of Mathematical Statistics (http://www.imstat.org)

References in corpus (8)

• Nearly unbiased variable selection under minimax concave penalty
• High-dimensional graphs and variable selection with the Lasso
• Simultaneous analysis of Lasso and Dantzig selector
• Pathwise coordinate optimization
• On the adaptive elastic-net with a diverging number of parameters
• One-step sparse estimates in nonconcave penalized likelihood models
• Coordinate descent algorithms for lasso penalized regression
• High-dimensional classification using features annealed independence rules

Cited by in corpus (2)

• ICR: Iterative Convex Refinement for Sparse Signal Recovery Using Spike and Slab Priors
• Nonconvex Optimization via MM Algorithms: Convergence Theory