Bayes and empirical-Bayes multiplicity adjustment in the variable-selection problem

Abstract

This paper studies the multiplicity-correction effect of standard Bayesian variable-selection priors in linear regression. Our first goal is to clarify when, and how, multiplicity correction happens automatically in Bayesian analysis, and to distinguish this correction from the Bayesian Ockham's-razor effect. Our second goal is to contrast empirical-Bayes and fully Bayesian approaches to variable selection through examples, theoretical results and simulations. Considerable differences between the two approaches are found. In particular, we prove a theorem that characterizes a surprising aymptotic discrepancy between fully Bayes and empirical Bayes. This discrepancy arises from a different source than the failure to account for hyperparameter uncertainty in the empirical-Bayes estimate. Indeed, even at the extreme, when the empirical-Bayes estimate converges asymptotically to the true variable-inclusion probability, the potential for a serious difference remains.

Published in at http://dx.doi.org/10.1214/10-AOS792 the Annals of Statistics (http://www.imstat.org/aos/) by the Institute of Mathematical Statistics (http://www.imstat.org)

References in corpus (4)

• Optimal predictive model selection
• Needles and straw in haystacks: Empirical Bayes estimates of possibly sparse sequences
• A comparison of the Benjamini-Hochberg procedure with some Bayesian rules for multiple testing
• Nonparametric Bayesian multiple testing for longitudinal performance stratification

Cited by in corpus (105)

• Inferring causal impact using Bayesian structural time-series models
• Criteria for Bayesian model choice with application to variable selection
• Bayesian linear regression with sparse priors
• Needles and Straw in a Haystack: Posterior concentration for possibly sparse sequences
• Overall Objective Priors
• DIAMONDS: a new Bayesian Nested Sampling tool. Application to Peak Bagging of solar-like oscillations
• The Horseshoe Estimator: Posterior Concentration around Nearly Black Vectors
• Bayesian Approaches to Copula Modelling
• Bayesian model search and multilevel inference for SNP association studies
• Mixtures of g-priors in Generalized Linear Models
• Asymptotic Bayes-optimality under sparsity of some multiple testing procedures
• Hyper-g Priors for Generalized Linear Models
• Simultaneous Variable and Covariance Selection with the Multivariate Spike-and-Slab Lasso
• Posterior contraction in sparse Bayesian factor models for massive covariance matrices
• Metallicity effect on stellar granulation detected from oscillating red giants in open clusters
• Nearly optimal Bayesian Shrinkage for High Dimensional Regression
• Spike-and-Slab Group Lassos for Grouped Regression and Sparse Generalized Additive Models
• Spike-and-Slab Meets LASSO: A Review of the Spike-and-Slab LASSO
• Tight conditions for consistency of variable selection in the context of high dimensionality
• Towards a framework for testing general relativity with extreme-mass-ratio-inspiral observations
• Posterior Model Consistency in Variable Selection as the Model Dimension Grows
• Scalable Bayesian Variable Selection Using Nonlocal Prior Densities in Ultrahigh-Dimensional Settings
• Bayesian Variable Selection For Survival Data Using Inverse Moment Priors
• Bayes Shrinkage at GWAS scale: Convergence and Approximation Theory of a Scalable MCMC Algorithm for the Horseshoe Prior
• Approximate Bayesian Model Selection with the Deviance Statistic
• Appropriate reduction of the posterior distribution in fully Bayesian inversions
• Efficient inference for genetic association studies with multiple outcomes
• Computational approaches for empirical Bayes methods and Bayesian sensitivity analysis
• Nonparametric Bayesian multiple testing for longitudinal performance stratification
• Improving forecasting performance using covariate-dependent copula models
• On the Beta Prime Prior for Scale Parameters in High-Dimensional Bayesian Regression Models
• MUCE: Bayesian Hierarchical Modeling for the Design and Analysis of Phase 1b Multiple Expansion Cohort Trials
• Quantifying Uncertainty in Transdimensional Markov Chain Monte Carlo Using Discrete Markov Models
• Shared kernel Bayesian screening
• Estimating causal structure using conditional DAG models
• Tight conditions for consistent variable selection in high dimensional nonparametric regression
• Bayesian $l_0$-regularized Least Squares
• Facial Keypoints Detection
• Inferring network structure from interventional time-course experiments
• A novel algorithmic approach to Bayesian Logic Regression
• High-dimensional variable selection via low-dimensional adaptive learning
• Model Selection and Multiple Testing - A Bayesian and Empirical Bayes Overview and some New Results
• Informed Bayesian T-Tests
• BVAR-Connect: A Variational Bayes Approach to Multi-Subject Vector Autoregressive Models for Inference on Brain Connectivity Networks
• Implicit Copulas from Bayesian Regularized Regression Smoothers
• Bayesian Genome- and Epigenome-wide Association Studies with Gene Level Dependence
• Optimal Bayesian design for model discrimination via classification
• Concentration of posterior probabilities and normalized L0 criteria
• The matryoshka doll prior: principled multiplicity correction in Bayesian model comparison
• Inconsistency identification in network meta-analysis via stochastic search variable selection
• Causal network inference using biochemical kinetics
• Scalable high-dimensional Bayesian varying coefficient models with unknown within-subject covariance
• Block Hyper-g Priors in Bayesian Regression
• Differentially private methods for managing model uncertainty in linear regression models
• Neuronized Priors for Bayesian Sparse Linear Regression
• Objective Bayesian Comparison of Constrained Analysis of Variance Models
• Flexible Bayesian Nonlinear Model Configuration
• Bayes and empirical Bayes: do they merge?
• Learning Bayesian networks: a copula approach for mixed-type data
• Methods and Tools for Bayesian Variable Selection and Model Averaging in Univariate Linear Regression
• Comment on García-Donato et al. (2025) "Model uncertainty and missing data: An objective Bayesian perspective"
• Improving the identification of antigenic sites in the H1N1 Influenza virus through accounting for the experimental structure in a sparse hierarchical Bayesian model
• A Bayesian model averaging approach for observational gene expression studies
• Bayesian Double Feature Allocation for Phenotyping with Electronic Health Records
• Mixtures of g-Priors for Generalised Additive Model Selection with Penalised Splines
• Modular Bayes screening for high-dimensional predictors
• Invited Discussion of "Model Uncertainty and Missing Data: An Objective Bayesian Perspective" by Gonzalo García-Donato , María Eugenia Castellanos , Stefano Cabras Alicia Quirós , and Anabel Forte
• Beyond Uniform Priors in Bayesian Network Structure Learning
• Learning from a lot: Empirical Bayes in high-dimensional prediction settings
• Model Selection for Maternal Hypertensive Disorders with Symmetric Hierarchical Dirichlet Processes
• Model Prior Distribution for Variable Selection in Linear Regression Models
• Towards a Multi-Subject Analysis of Neural Connectivity
• Quantile Graphical Models: Bayesian Approaches
• Bayesian Variable Selection with Structure Learning: Applications in Integrative Genomics
• A Latent Space Model for Cognitive Social Structures Data
• A Metropolized adaptive subspace algorithm for high-dimensional Bayesian variable selection
• Sketching in Bayesian High Dimensional Regression With Big Data Using Gaussian Scale Mixture Priors
• Inference for multiple treatment effects using confounder importance learning
• Variations of Power-Expected-Posterior Priors in Normal Regression Models
• A Minimum Message Length Criterion for Robust Linear Regression
• The scalable Birth-Death MCMC Algorithm for Mixed Graphical Model Learning with Application to Genomic Data Integration
• Irreproducibility; Nothing is More Predictable
• Empirical Bayes Model Averaging with Influential Observations: Tuning Zellner's g Prior for Predictive Robustness
• Laplace Approximation in High-dimensional Bayesian Regression
• A deconvolution path for mixtures
• Bayesian Edge Regression in Undirected Graphical Models to Characterize Interpatient Heterogeneity in Cancer
• Comment on Article by Berger, Bernardo, and Sun
• Adaptive posterior convergence in sparse high dimensional clipped generalized linear models
• The Inuence of Misspecified Covariance on False Discovery Control when Using Posterior Probabilities
• Disjunct Support Spike and Slab Priors for Variable Selection in Regression under Quasi-sparseness
• A New Step-down Procedure for Simultaneous Hypothesis Testing Under Dependence
• Screening the Discrepancy Function of a Computer Model
• Handling Factors in Variable Selection Problems
• An approach for finding fully Bayesian optimal designs using normal-based approximations to loss functions
• Exploring dependence between categorical variables: benefits and limitations of using variable selection within Bayesian clustering in relation to log-linear modelling with interaction terms
• Bayesian Variable Selection on Model Spaces Constrained by Heredity Conditions
• Prioritizing covariates in the planning of future studies in the meta-analytic framework
• Bayesian $L_{\frac{1}{2}}$ regression
• Bayesian Multivariate Probability of Success Using Historical Data with Strict Control of Family-wise Error Rate
• Power-Expected-Posterior Priors as Mixtures of g-Priors
• Scalable Bayesian variable selection and model averaging under block orthogonal design
• Flexible Bayesian Multiple Comparison Adjustment Using Dirichlet Process and Beta-Binomial Model Priors
• Some Permutationllay Symmetric Multiple Hypotheses Testing Rules Under Dependent Set up
• Bayesian model selection in additive partial linear models via locally adaptive splines
• Bayesian Matrix Completion for Hypothesis Testing