A comparison of learning rate selection methods in generalized Bayesian inference

Abstract

Generalized Bayes posterior distributions are formed by putting a fractional power on the likelihood before combining with the prior via Bayes's formula. This fractional power, which is often viewed as a remedy for potential model misspecification bias, is called the learning rate, and a number of data-driven learning rate selection methods have been proposed in the recent literature. Each of these proposals has a different focus, a different target they aim to achieve, which makes them difficult to compare. In this paper, we provide a direct head-to-head comparison of these learning rate selection methods in various misspecified model scenarios, in terms of several relevant metrics, in particular, coverage probability of the generalized Bayes credible regions. In some examples all the methods perform well, while in others the misspecification is too severe to be overcome, but we find that the so-called generalized posterior calibration algorithm tends to outperform the others in terms of credible region coverage probability.

arXiv comment22 pages, 2 figures, 4 tables

References in corpus (18)

• A General Framework for Updating Belief Distributions
• From $\epsilon$-entropy to KL-entropy: Analysis of minimum information complexity density estimation
• Misspecification in infinite-dimensional Bayesian statistics
• Is Bayes Posterior just Quick and Dirty Confidence?
• Calibrating general posterior credible regions
• Empirical Bayes posterior concentration in sparse high-dimensional linear models
• Fast Rates for General Unbounded Loss Functions: from ERM to Generalized Bayes
• False confidence, non-additive beliefs, and valid statistical inference
• On Posterior Concentration in Misspecified Models
• Gibbs posterior concentration rates under sub-exponential type losses
• Gibbs posterior inference on the minimum clinically important difference
• Model-free posterior inference on the area under the receiver operating characteristic curve
• Safe-Bayesian Generalized Linear Regression
• Data-driven priors and their posterior concentration rates
• Gibbs posterior inference on multivariate quantiles
• Empirical priors and coverage of posterior credible sets in a sparse normal mean model
• Robust and rate-optimal Gibbs posterior inference on the boundary of a noisy image
• Empirical priors for prediction in sparse high-dimensional linear regression

Cited by in corpus (10)

• Gibbs posterior concentration rates under sub-exponential type losses
• Gibbs posterior inference on multivariate quantiles
• Many wrong models approach to localize an odor source in turbulence with static sensors
• Concentration properties of fractional posterior in 1-bit matrix completion
• Concentration of a sparse Bayesian model with Horseshoe prior in estimating high-dimensional precision matrix
• Sampling algorithms in statistical physics: a guide for statistics and machine learning
• Calibrating generalized predictive distributions
• Semiparametric inference using fractional posteriors
• Generalized Posterior Calibration via Sequential Monte Carlo Sampler
• Weighted Particle-Based Optimization for Efficient Generalized Posterior Calibration