Calibrating generalized predictive distributions

Abstract

In prediction problems, it is common to model the data-generating process and then use a model-based procedure, such as a Bayesian predictive distribution, to quantify uncertainty about the next observation. However, if the posited model is misspecified, then its predictions may not be calibrated -- that is, the predictive distribution's quantiles may not be nominal frequentist prediction upper limits, even asymptotically. Rather than abandoning the comfort of a model-based formulation for a more complicated non-model-based approach, here we propose a strategy in which the data itself helps determine if the assumed model-based solution should be adjusted to account for model misspecification. This is achieved through a generalized Bayes formulation where a learning rate parameter is tuned, via the proposed generalized predictive calibration (GPrC) algorithm, to make the predictive distribution calibrated, even under model misspecification. Extensive numerical experiments are presented, under a variety of settings, demonstrating the proposed GPrC algorithm's validity, efficiency, and robustness.

arXiv comment33 pages, 5 figures, 6 tables

References in corpus (11)

• From $\epsilon$-entropy to KL-entropy: Analysis of minimum information complexity density estimation
• Misspecification in infinite-dimensional Bayesian statistics
• Calibrating general posterior credible regions
• Fast Rates for General Unbounded Loss Functions: from ERM to Generalized Bayes
• On recursive Bayesian predictive distributions
• A comparison of learning rate selection methods in generalized Bayesian inference
• On Posterior Concentration in Misspecified Models
• Gibbs posterior concentration rates under sub-exponential type losses
• Diagnosing model misspecification and performing generalized Bayes' updates via probabilistic classifiers
• On nonparametric estimation of a mixing density via the predictive recursion algorithm
• Methods to Compute Prediction Intervals: A Review and New Results

Cited by in corpus (1)

• Gibbs posterior concentration rates under sub-exponential type losses