Neuronized Priors for Bayesian Sparse Linear Regression
arXiv:1810.00141
Abstract
Although Bayesian variable selection methods have been intensively studied, their routine use in practice has not caught up with their non-Bayesian counterparts such as Lasso, likely due to difficulties in both computations and flexibilities of prior choices. To ease these challenges, we propose the neuronized priors to unify and extend some popular shrinkage priors, such as Laplace, Cauchy, horseshoe, and spike-and-slab priors. A neuronized prior can be written as the product of a Gaussian weight variable and a scale variable transformed from Gaussian via an activation function. Compared with classic spike-and-slab priors, the neuronized priors achieve the same explicit variable selection without employing any latent indicator variables, which results in both more efficient and flexible posterior sampling and more effective posterior modal estimation. Theoretically, we provide specific conditions on the neuronized formulation to achieve the optimal posterior contraction rate, and show that a broadly applicable MCMC algorithm achieves an exponentially fast convergence rate under the neuronized formulation. We also examine various simulated and real data examples and demonstrate that using the neuronization representation is computationally more or comparably efficient than its standard counterpart in all well-known cases. An R package NPrior is provided in the CRAN for using neuronized priors in Bayesian linear regression.
References in corpus (10)
- General state space Markov chains and MCMC algorithms
- On the conditions used to prove oracle results for the Lasso
- Bayes and empirical-Bayes multiplicity adjustment in the variable-selection problem
- Bayesian variable selection with shrinking and diffusing priors
- Needles and Straw in a Haystack: Posterior concentration for possibly sparse sequences
- Nearly optimal Bayesian Shrinkage for High Dimensional Regression
- Variable transformation to obtain geometric ergodicity in the random-walk Metropolis algorithm
- Variance prior forms for high-dimensional Bayesian variable selection
- Kernel estimators of asymptotic variance for adaptive Markov chain Monte Carlo
- Parameter expansion in local-shrinkage models