Gibbs posterior concentration rates under sub-exponential type losses

Abstract

Bayesian posterior distributions are widely used for inference, but their dependence on a statistical model creates some challenges. In particular, there may be lots of nuisance parameters that require prior distributions and posterior computations, plus a potentially serious risk of model misspecification bias. Gibbs posterior distributions, on the other hand, offer direct, principled, probabilistic inference on quantities of interest through a loss function, not a model-based likelihood. Here we provide simple sufficient conditions for establishing Gibbs posterior concentration rates when the loss function is of a sub-exponential type. We apply these general results in a range of practically relevant examples, including mean regression, quantile regression, and sparse high-dimensional classification. We also apply these techniques in an important problem in medical statistics, namely, estimation of a personalized minimum clinically important difference.

arXiv comment59 pages, 1 figure

References in corpus (24)

• Rényi Divergence and Kullback-Leibler Divergence
• An MCMC Approach to Classical Estimation
• 2004 IMS Medallion Lecture: Local Rademacher complexities and oracle inequalities in risk minimization
• A General Framework for Updating Belief Distributions
• Risk bounds for statistical learning
• Asymptotics for minimisers of convex processes
• Misspecification in infinite-dimensional Bayesian statistics
• Gibbs posterior for variable selection in high-dimensional classification and data mining
• Calibrating general posterior credible regions
• On the properties of variational approximations of Gibbs posteriors
• Empirical Bayes posterior concentration in sparse high-dimensional linear models
• A Primer on PAC-Bayesian Learning
• False confidence, non-additive beliefs, and valid statistical inference
• PAC-Bayesian Bounds for Randomized Empirical Risk Minimizers
• A comparison of learning rate selection methods in generalized Bayesian inference
• On Posterior Concentration in Misspecified Models
• Assigning a value to a power likelihood in a general Bayesian model
• Gibbs posterior concentration rates under sub-exponential type losses
• Gibbs posterior inference on the minimum clinically important difference
• Model-free posterior inference on the area under the receiver operating characteristic curve
• Gibbs posterior inference on multivariate quantiles
• Robust and rate-optimal Gibbs posterior inference on the boundary of a noisy image
• Calibrating generalized predictive distributions
• A note on Bayesian convergence rates under local prior support conditions

Cited by in corpus (15)

• User-friendly introduction to PAC-Bayes bounds
• A comparison of learning rate selection methods in generalized Bayesian inference
• Forecasting: theory and practice
• Gibbs posterior concentration rates under sub-exponential type losses
• Gibbs posterior inference on multivariate quantiles
• Probabilistic Contrastive Principal Component Analysis
• Adaptive variational Bayes: Optimality, computation and applications
• Empirical Bayes inference in sparse high-dimensional generalized linear models
• Bernstein - von Mises theorem and misspecified models: a review
• Sampling algorithms in statistical physics: a guide for statistics and machine learning
• Calibrating generalized predictive distributions
• Statistical Inference for Bayesian Risk Minimization via Exponentially Tilted Empirical Likelihood
• Generalized Bayes Approach to Inverse Problems with Model Misspecification
• Generalized Posterior Calibration via Sequential Monte Carlo Sampler
• Weighted Particle-Based Optimization for Efficient Generalized Posterior Calibration