Gibbs posterior concentration rates under sub-exponential type losses

Abstract

Bayesian posterior distributions are widely used for inference, but their dependence on a statistical model creates some challenges. In particular, there may be lots of nuisance parameters that require prior distributions and posterior computations, plus a potentially serious risk of model misspecification bias. Gibbs posterior distributions, on the other hand, offer direct, principled, probabilistic inference on quantities of interest through a loss function, not a model-based likelihood. Here we provide simple sufficient conditions for establishing Gibbs posterior concentration rates when the loss function is of a sub-exponential type. We apply these general results in a range of practically relevant examples, including mean regression, quantile regression, and sparse high-dimensional classification. We also apply these techniques in an important problem in medical statistics, namely, estimation of a personalized minimum clinically important difference.

59 pages, 1 figure

References in corpus (10)

• An MCMC Approach to Classical Estimation
• Asymptotics for minimisers of convex processes
• Misspecification in infinite-dimensional Bayesian statistics
• Gibbs posterior for variable selection in high-dimensional classification and data mining
• On the properties of variational approximations of Gibbs posteriors
• PAC-Bayesian Bounds for Randomized Empirical Risk Minimizers
• A comparison of learning rate selection methods in generalized Bayesian inference
• Assigning a value to a power likelihood in a general Bayesian model
• Gibbs posterior concentration rates under sub-exponential type losses
• Calibrating generalized predictive distributions

Cited by in corpus (14)

• User-friendly introduction to PAC-Bayes bounds
• A comparison of learning rate selection methods in generalized Bayesian inference
• Forecasting: theory and practice
• Gibbs posterior concentration rates under sub-exponential type losses
• Probabilistic Contrastive Principal Component Analysis
• Adaptive variational Bayes: Optimality, computation and applications
• Empirical Bayes inference in sparse high-dimensional generalized linear models
• Bernstein - von Mises theorem and misspecified models: a review
• Sampling algorithms in statistical physics: a guide for statistics and machine learning
• Calibrating generalized predictive distributions
• Statistical Inference for Bayesian Risk Minimization via Exponentially Tilted Empirical Likelihood
• Generalized Bayes Approach to Inverse Problems with Model Misspecification
• Generalized Posterior Calibration via Sequential Monte Carlo Sampler
• Weighted Particle-Based Optimization for Efficient Generalized Posterior Calibration