Data-driven priors and their posterior concentration rates

Abstract

In high-dimensional problems, choosing a prior distribution such that the corresponding posterior has desirable practical and theoretical properties can be challenging. This begs the question: can the data be used to help choose a good prior? In this paper, we develop a general strategy for constructing a data-driven or empirical prior and sufficient conditions for the corresponding posterior distribution to achieve a certain concentration rate. The idea is that the prior should put sufficient mass on parameter values for which the likelihood is large. An interesting byproduct of this data-driven centering is that the asymptotic properties of the posterior are less sensitive to the prior shape which, in turn, allows users to work with priors of computationally convenient forms while maintaining the desired rates. General results on both adaptive and non-adaptive rates based on empirical priors are presented, along with illustrations in density estimation, nonparametric regression, and high-dimensional structured normal models.

31 pages

References in corpus (10)

• Needles and Straw in a Haystack: Posterior concentration for possibly sparse sequences
• Convergence rates of posterior distributions for noniid observations
• Adaptive Bayesian estimation using a Gaussian random field with inverse Gamma bandwidth
• Posterior convergence rates of Dirichlet mixtures at smooth densities
• Contributed Discussion to Uncertainty Quantification for the Horseshoe by Stéphanie van der Pas, Botond Szabó and Aad van der Vaart
• On rates of convergence for posterior distributions in infinite-dimensional models
• Empirical priors and coverage of posterior credible sets in a sparse normal mean model
• Empirical priors for prediction in sparse high-dimensional linear regression
• Empirical priors and posterior concentration rates for a monotone density
• Minimax Posterior Convergence Rates and Model Selection Consistency in High-dimensional DAG Models based on Sparse Cholesky Factors

Cited by in corpus (6)

• A comparison of learning rate selection methods in generalized Bayesian inference
• Empirical priors and coverage of posterior credible sets in a sparse normal mean model
• An empirical $G$-Wishart prior for sparse high-dimensional Gaussian graphical models
• Empirical Bayes inference in sparse high-dimensional generalized linear models
• Adaptive posterior contraction rates for empirical Bayesian drift estimation of a diffusion
• Convergence Rates of Empirical Bayes Posterior Distributions: A Variational Perspective