Bayesian model choice and information criteria in sparse generalized linear models

Abstract

We consider Bayesian model selection in generalized linear models that are high-dimensional, with the number of covariates p being large relative to the sample size n, but sparse in that the number of active covariates is small compared to p. Treating the covariates as random and adopting an asymptotic scenario in which p increases with n, we show that Bayesian model selection using certain priors on the set of models is asymptotically equivalent to selecting a model using an extended Bayesian information criterion. Moreover, we prove that the smallest true model is selected by either of these methods with probability tending to one. Having addressed random covariates, we are also able to give a consistency result for pseudo-likelihood approaches to high-dimensional sparse graphical modeling. Experiments on real data demonstrate good performance of the extended Bayesian information criterion for regression and for graphical models.

References in corpus (6)

• On the "degrees of freedom" of the lasso
• Stability Approach to Regularization Selection (StARS) for High Dimensional Graphical Models
• Consistency of objective Bayes factors as the model dimension grows
• Selection Consistency of EBIC for GLIM with Non-canonical Links and Diverging Number of Parameters
• Bayes Variable Selection in Semiparametric Linear Models
• Consistency of Bayesian Linear Model Selection With a Growing Number of Parameters

Cited by in corpus (7)

• A Tutorial on Regularized Partial Correlation Networks
• Estimating Psychological Networks and their Accuracy: A Tutorial Paper
• mgm: Estimating Time-Varying Mixed Graphical Models in High-Dimensional Data
• Towards a Multi-Subject Analysis of Neural Connectivity
• Local Whittle estimation of high-dimensional long-run variance and precision matrices
• Robust Variable and Interaction Selection for Logistic Regression and Multiple Index Models
• Logistic regression and Ising networks: prediction and estimation when violating lasso assumptions