Unbiased estimation of log normalizing constants with applications to Bayesian cross-validation

Abstract

Posterior distributions often feature intractable normalizing constants, called marginal likelihoods or evidence, that are useful for model comparison via Bayes factors. This has motivated a number of methods for estimating ratios of normalizing constants in statistics. In computational physics the logarithm of these ratios correspond to free energy differences. Combining unbiased Markov chain Monte Carlo estimators with path sampling, also called thermodynamic integration, we propose new unbiased estimators of the logarithm of ratios of normalizing constants. As a by-product, we propose unbiased estimators of the Bayesian cross-validation criterion. The proposed estimators are consistent, asymptotically Normal and can easily benefit from parallel processing devices. Various examples are considered for illustration.

arXiv comment19 pages, 6 figures

References in corpus (3)

• Unbiased Markov chain Monte Carlo with couplings
• On the Pitfalls of Nested Monte Carlo
• Sampling normalizing constants in high dimensions using inhomogeneous diffusions

Cited by in corpus (8)

• Stochastic Normalizing Flows
• Implicitly Adaptive Importance Sampling
• Unbiased Markov chain Monte Carlo with couplings
• Robust leave-one-out cross-validation for high-dimensional Bayesian models
• Distributed Computation for Marginal Likelihood based Model Choice
• Parallel Tempering on Optimized Paths
• Adaptive Path Sampling in Metastable Posterior Distributions
• Multilevel Estimation of Normalization Constants Using the Ensemble Kalman-Bucy Filter