Leave-One-Out Cross-Validation for Bayesian Model Comparison in Large Data
arXiv:2001.00980
Abstract
Recently, new methods for model assessment, based on subsampling and posterior approximations, have been proposed for scaling leave-one-out cross-validation (LOO) to large datasets. Although these methods work well for estimating predictive performance for individual models, they are less powerful in model comparison. We propose an efficient method for estimating differences in predictive performance by combining fast approximate LOO surrogates with exact LOO subsampling using the difference estimator and supply proofs with regards to scaling characteristics. The resulting approach can be orders of magnitude more efficient than previous approaches, as well as being better suited to model comparison.
References in corpus (8)
- Practical Bayesian model evaluation using leave-one-out cross-validation and WAIC
- Asymptotic Equivalence of Bayes Cross Validation and Widely Applicable Information Criterion in Singular Learning Theory
- A Conceptual Introduction to Hamiltonian Monte Carlo
- Sparsity information and regularization in the horseshoe and other shrinkage priors
- Pareto Smoothed Importance Sampling
- Projective Inference in High-dimensional Problems: Prediction and Feature Selection
- Bayesian leave-one-out cross-validation approximations for Gaussian latent variable models
- A Swiss Army Infinitesimal Jackknife