On the Beta Prime Prior for Scale Parameters in High-Dimensional Bayesian Regression Models

Abstract

We study high-dimensional Bayesian linear regression with a general beta prime distribution for the scale parameter. Under the assumption of sparsity, we show that appropriate selection of the hyperparameters in the beta prime prior leads to the (near) minimax posterior contraction rate when $p\gg n$. For finite samples, we propose a data-adaptive method for estimating the hyperparameters based on marginal maximum likelihood (MML). This enables our prior to adapt to both sparse and dense settings, and under our proposed empirical Bayes procedure, the MML estimates are never at risk of collapsing to zero. We derive efficient Monte Carlo EM and variational EM algorithms for implementing our model, which are available in the R package NormalBetaPrime. Simulations and analysis of a gene expression data set illustrate our model's self-adaptivity to varying levels of sparsity and signal strengths.

37 pages, 4 figures, 3 tables. We have added a section on posterior computation and corrected the theoretical results. Sections on normal means estimation were removed in this updated technical report

References in corpus (5)

• Nearly unbiased variable selection under minimax concave penalty
• Bayes and empirical-Bayes multiplicity adjustment in the variable-selection problem
• Bayesian variable selection with shrinking and diffusing priors
• Nearly optimal Bayesian Shrinkage for High Dimensional Regression
• Bayes Shrinkage at GWAS scale: Convergence and Approximation Theory of a Scalable MCMC Algorithm for the Horseshoe Prior

Cited by in corpus (2)

• Spike-and-Slab Meets LASSO: A Review of the Spike-and-Slab LASSO
• Gibbs Sampling using Anti-correlation Gaussian Data Augmentation, with Applications to L1-ball-type Models