Bayesian fractional posteriors

Abstract

We consider the fractional posterior distribution that is obtained by updating a prior distribution via Bayes theorem with a fractional likelihood function, a usual likelihood function raised to a fractional power. First, we analyze the contraction property of the fractional posterior in a general misspecified framework. Our contraction results only require a prior mass condition on certain Kullback-Leibler (KL) neighborhood of the true parameter (or the KL divergence minimizer in the misspecified case), and obviate constructions of test functions and sieves commonly used in the literature for analyzing the contraction property of a regular posterior. We show through a counterexample that some condition controlling the complexity of the parameter space is necessary for the regular posterior to contract, rendering additional flexibility on the choice of the prior for the fractional posterior. Second, we derive a novel Bayesian oracle inequality based on a PAC-Bayes inequality in misspecified models. Our derivation reveals several advantages of averaging based Bayesian procedures over optimization based frequentist procedures. As an application of the Bayesian oracle inequality, we derive a sharp oracle inequality in the convex regression problem under an arbitrary dimension. We also illustrate the theory in Gaussian process regression and density estimation problems.

35 pages

References in corpus (11)

• Rényi Divergence and Kullback-Leibler Divergence
• An MCMC Approach to Classical Estimation
• 2004 IMS Medallion Lecture: Local Rademacher complexities and oracle inequalities in risk minimization
• Adaptive Bayesian estimation using a Gaussian random field with inverse Gamma bandwidth
• Adaptive Bayesian multivariate density estimation with Dirichlet mixtures
• On risk bounds in isotonic and other shape restricted regression problems
• Gibbs posterior for variable selection in high-dimensional classification and data mining
• A new perspective on least squares under convex constraint
• Asymptotically minimax empirical Bayes estimation of a sparse normal mean vector
• Empirical entropy, minimax regret and minimax risk
• Bayesian nonparametric multivariate convex regression

Cited by in corpus (5)

• Consistency of Variational Bayes Inference for Estimation and Model Selection in Mixtures
• Bayes Shrinkage at GWAS scale: Convergence and Approximation Theory of a Scalable MCMC Algorithm for the Horseshoe Prior
• Convergence Rates of Variational Inference in Sparse Deep Learning
• A Generalization Bound for Online Variational Inference
• Mini-batch Tempered MCMC