Using stacking to average Bayesian predictive distributions
arXiv:1704.02030 · doi:10.1214/17-BA1091
Abstract
The widely recommended procedure of Bayesian model averaging is flawed in the M-open setting in which the true data-generating process is not one of the candidate models being fit. We take the idea of stacking from the point estimation literature and generalize to the combination of predictive distributions, extending the utility function to any proper scoring rule, using Pareto smoothed importance sampling to efficiently compute the required leave-one-out posterior distributions and regularization to get more stability. We compare stacking of predictive distributions to several alternatives: stacking of means, Bayesian model averaging (BMA), pseudo-BMA using AIC-type weighting, and a variant of pseudo-BMA that is stabilized using the Bayesian bootstrap. Based on simulations and real-data applications, we recommend stacking of predictive distributions, with BB-pseudo-BMA as an approximate alternative when computation cost is an issue.
References in corpus (7)
- Practical Bayesian model evaluation using leave-one-out cross-validation and WAIC
- Variational Inference: A Review for Statisticians
- Asymptotic Equivalence of Bayes Cross Validation and Widely Applicable Information Criterion in Singular Learning Theory
- Issues in Stacked Generalization
- Using stacking to average Bayesian predictive distributions
- Comparison of Bayesian predictive methods for model selection
- Automatic Differentiation Variational Inference
Cited by in corpus (44)
- Prediction-Based Decisions and Fairness: A Catalogue of Choices, Assumptions, and Definitions
- Using stacking to average Bayesian predictive distributions
- Relativistic redshift of the star S0-2 orbiting the Galactic center supermassive black hole
- A review of predictive uncertainty estimation with machine learning
- Stacking Models for Nearly Optimal Link Prediction in Complex Networks
- KiDS-1000 Cosmology: constraints beyond flat CDM
- Projective Inference in High-dimensional Problems: Prediction and Feature Selection
- Bayesian comparison of latent variable models: Conditional vs marginal likelihoods
- Approximate leave-future-out cross-validation for Bayesian time series models
- Boba: Authoring and Visualizing Multiverse Analyses
- Bayesian Model-Averaged Meta-Analysis in Medicine
- Bayesian forecasting of multivariate time series: Scalability, structure uncertainty and decisions
- Towards Precise and Accurate Calculations of Neutrinoless Double-Beta Decay: Project Scoping Workshop Report
- Bayesian hierarchical stacking: Some models are (somewhere) useful
- The Magnificent Five Images of Supernova Refsdal: Time Delay and Magnification Measurements
- A framework for probabilistic weather forecast post-processing across models and lead times using machine learning
- Exploring physics of ferroelectric domain walls via Bayesian analysis of atomically resolved STEM data
- Merging satellite and gauge-measured precipitation using LightGBM with an emphasis on extreme quantiles
- Bayesian Median Autoregression for Robust Time Series Forecasting
- Hot Rocks Survey II: The thermal emission of TOI-1468 b reveals a hot bare rock
- Projecting UK Mortality using Bayesian Generalised Additive Models
- Robust leave-one-out cross-validation for high-dimensional Bayesian models
- Weak evidence for variable occultation depth of 55 Cnc e with TESS
- Some models are useful, but how do we know which ones? Towards a unified Bayesian model taxonomy
- Holes in Bayesian Statistics
- Adaptive Variable Selection for Sequential Prediction in Multivariate Dynamic Models
- Combining probability distributions: Extending the logarithmic pooling approach
- Bayesian inference in band excitation Scanning Probe Microscopy for optimal dynamic model selection in imaging
- Assessing Bayes factor surfaces using interactive visualization and computer surrogate modeling
- A Bayesian semi-parametric approach for modeling memory decay in dynamic social networks
- Ensemble learning for blending gridded satellite and gauge-measured precipitation data
- Uncertainty in Bayesian Leave-One-Out Cross-Validation Based Model Comparison
- Interpolating between small- and large- expansions using Bayesian Model Mixing
- Assessing inter-rater reliability with heterogeneous variance components models: Flexible approach accounting for contextual variables
- Does Bayesian Model Averaging improve polynomial extrapolations? Two toy problems as tests
- Observations of scattered light from exoplanet atmospheres
- A loss discounting framework for model averaging and selection in time series models
- A review of machine learning concepts and methods for addressing challenges in probabilistic hydrological post-processing and forecasting
- A generalized statistical model for fits to parton distributions
- Fusion of Gaussian Processes Predictions with Monte Carlo Sampling
- Combinations of distributional regression algorithms with application in uncertainty estimation of corrected satellite precipitation products
- Predicting competitions by combining conditional logistic regression and subjective Bayes: An Academy Awards case study
- Ensemble learning for uncertainty estimation with application to the correction of satellite precipitation products
- Automating Model Comparison in Factor Graphs