Bayesian leave-one-out cross-validation approximations for Gaussian latent variable models

Abstract

The future predictive performance of a Bayesian model can be estimated using Bayesian cross-validation. In this article, we consider Gaussian latent variable models where the integration over the latent values is approximated using the Laplace method or expectation propagation (EP). We study the properties of several Bayesian leave-one-out (LOO) cross-validation approximations that in most cases can be computed with a small additional cost after forming the posterior approximation given the full data. Our main objective is to assess the accuracy of the approximative LOO cross-validation estimators. That is, for each method (Laplace and EP) we compare the approximate fast computation with the exact brute force LOO computation. Secondarily, we evaluate the accuracy of the Laplace and EP approximations themselves against a ground truth established through extensive Markov chain Monte Carlo simulation. Our empirical results show that the approach based upon a Gaussian approximation to the LOO marginal distribution (the so-called cavity distribution) gives the most accurate and reliable results among the fast methods.

References in corpus (9)

• Auto-Encoding Variational Bayes
• Practical Bayesian model evaluation using leave-one-out cross-validation and WAIC
• Asymptotic Equivalence of Bayes Cross Validation and Widely Applicable Information Criterion in Singular Learning Theory
• Comparison of Bayesian predictive methods for model selection
• Pareto Smoothed Importance Sampling
• Slice sampling covariance hyperparameters of latent Gaussian models
• Approximating Cross-validatory Predictive Evaluation in Bayesian Latent Variables Models with Integrated IS and WAIC
• Modelling local and global phenomena with sparse Gaussian processes
• Nested Expectation Propagation for Gaussian Process Classification with a Multinomial Probit Likelihood

Cited by in corpus (11)

• Practical Bayesian model evaluation using leave-one-out cross-validation and WAIC
• Comparison of Bayesian predictive methods for model selection
• Bayesian comparison of latent variable models: Conditional vs marginal likelihoods
• Leave-One-Out Cross-Validation for Bayesian Model Comparison in Large Data
• Time series forecasting with Gaussian Processes needs priors
• Additive multivariate Gaussian processes for joint species distribution modeling with heterogeneous data
• Instrumental Variable Regression via Kernel Maximum Moment Loss
• Error bounds in estimating the out-of-sample prediction error using leave-one-out cross validation in high-dimensions
• Information Criterion for Boltzmann Approximation Problems
• Approximate Cross-validated Mean Estimates for Bayesian Hierarchical Regression Models
• Bayesian model selection in the $\mathcal{M}$-open setting -- Approximate posterior inference and probability-proportional-to-size subsampling for efficient large-scale leave-one-out cross-validation