Bayesian hierarchical stacking: Some models are (somewhere) useful
arXiv:2101.08954 · doi:10.1214/21-BA1287
Abstract
Stacking is a widely used model averaging technique that asymptotically yields optimal predictions among linear averages. We show that stacking is most effective when model predictive performance is heterogeneous in inputs, and we can further improve the stacked mixture with a hierarchical model. We generalize stacking to Bayesian hierarchical stacking. The model weights are varying as a function of data, partially-pooled, and inferred using Bayesian inference. We further incorporate discrete and continuous inputs, other structured priors, and time series and longitudinal data. To verify the performance gain of the proposed method, we derive theory bounds, and demonstrate on several applied problems.
arXiv commentminor revision
References in corpus (12)
- Practical Bayesian model evaluation using leave-one-out cross-validation and WAIC
- Using stacking to average Bayesian predictive distributions
- Comparison of Bayesian predictive methods for model selection
- Pareto Smoothed Importance Sampling
- Feature-Weighted Linear Stacking
- Stacking Models for Nearly Optimal Link Prediction in Complex Networks
- Bayesian Hierarchical Mixtures of Experts
- Approximate leave-future-out cross-validation for Bayesian time series models
- Testing hypotheses via a mixture estimation model
- Stacking for Non-mixing Bayesian Computations: The Curse and Blessing of Multimodal Posteriors
- Minimax Optimal Bayesian Aggregation
- Reducing the error of Monte Carlo Algorithms by Learning Control Variates