Variational approximations of empirical Bayes posteriors in high-dimensional linear models

Abstract

In high-dimensions, the prior tails can have a significant effect on both posterior computation and asymptotic concentration rates. To achieve optimal rates while keeping the posterior computations relatively simple, an empirical Bayes approach has recently been proposed, featuring thin-tailed conjugate priors with data-driven centers. While conjugate priors ease some of the computational burden, Markov chain Monte Carlo methods are still needed, which can be expensive when dimension is high. In this paper, we develop a variational approximation to the empirical Bayes posterior that is fast to compute and retains the optimal concentration rate properties of the original. In simulations, our method is shown to have superior performance compared to existing variational approximations in the literature across a wide range of high-dimensional settings.

arXiv comment30 pages, 1 figure, 4 tables

References in corpus (8)

• Empirical Bayes selection of wavelet thresholds
• Needles and Straw in a Haystack: Posterior concentration for possibly sparse sequences
• Frequentist Consistency of Variational Bayes
• Asymptotically minimax empirical Bayes estimation of a sparse normal mean vector
• A nonparametric empirical Bayes framework for large-scale multiple testing
• A Variational Algorithm for Bayesian Variable Selection
• An empirical $G$-Wishart prior for sparse high-dimensional Gaussian graphical models
• Bayesian inference in high-dimensional linear models using an empirical correlation-adaptive prior

Cited by in corpus (3)

• Variational Bayes for high-dimensional linear regression with sparse priors
• Adaptive variational Bayes: Optimality, computation and applications
• Asymptotically optimal inference in sparse sequence models with a simple data-dependent measure