Consistency of Bayesian procedures for variable selection

Abstract

It has long been known that for the comparison of pairwise nested models, a decision based on the Bayes factor produces a consistent model selector (in the frequentist sense). Here we go beyond the usual consistency for nested pairwise models, and show that for a wide class of prior distributions, including intrinsic priors, the corresponding Bayesian procedure for variable selection in normal regression is consistent in the entire class of normal linear models. We find that the asymptotics of the Bayes factors for intrinsic priors are equivalent to those of the Schwarz (BIC) criterion. Also, recall that the Jeffreys--Lindley paradox refers to the well-known fact that a point null hypothesis on the normal mean parameter is always accepted when the variance of the conjugate prior goes to infinity. This implies that some limiting forms of proper prior distributions are not necessarily suitable for testing problems. Intrinsic priors are limits of proper prior distributions, and for finite sample sizes they have been proved to behave extremely well for variable selection in regression; a consequence of our results is that for intrinsic priors Lindley's paradox does not arise.

Published in at http://dx.doi.org/10.1214/08-AOS606 the Annals of Statistics (http://www.imstat.org/aos/) by the Institute of Mathematical Statistics (http://www.imstat.org)

References in corpus (1)

• Optimal predictive model selection

Cited by in corpus (13)

• Criteria for Bayesian model choice with application to variable selection
• Consistency of objective Bayes factors as the model dimension grows
• Model Selection Principles in Misspecified Models
• Model Order Selection Based on Information Theoretic Criteria: Design of the Penalty
• Power-Expected-Posterior Priors for Variable Selection in Gaussian Linear Models
• Posterior Model Consistency in Variable Selection as the Model Dimension Grows
• Simultaneous SNP identification in association studies with missing data
• The Whetstone and the Alum Block: Balanced Objective Bayesian Comparison of Nested Models for Discrete Data
• Exploring the single-pulse behaviours of PSR J0628+0909 with FAST
• Consistency of Bayes factor for nonnested model selection when the model dimension grows
• Informed reversible jump algorithms
• On Statistical Efficiency in Learning
• Objective Bayesian Comparison of Constrained Analysis of Variance Models