Bayesian model comparison for simulation-based inference

Abstract

Comparison of appropriate models to describe observational data is a fundamental task of science. The Bayesian model evidence, or marginal likelihood, is a computationally challenging, yet crucial, quantity to estimate to perform Bayesian model comparison. We introduce a methodology to compute the Bayesian model evidence in simulation-based inference (SBI) scenarios (also often called likelihood-free inference). In particular, we leverage the recently proposed learnt harmonic mean estimator and exploit the fact that it is decoupled from the method used to generate posterior samples, i.e. it requires posterior samples only, which may be generated by any approach. This flexibility, which is lacking in many alternative methods for computing the model evidence, allows us to develop SBI model comparison techniques for the three main neural density estimation approaches, including neural posterior estimation (NPE), neural likelihood estimation (NLE), and neural ratio estimation (NRE). We demonstrate and validate our SBI evidence calculation techniques on a range of inference problems, including a gravitational wave example. Moreover, we further validate the accuracy of the learnt harmonic mean estimator, implemented in the HARMONIC software, in likelihood-based settings. These results highlight the potential of HARMONIC as a sampler-agnostic method to estimate the model evidence in both likelihood-based and simulation-based scenarios.

13 pages, 5 figures. 2 min. Matches version published in RASTI. Summary video available at https://youtu.be/xbTS_5pGjaA. HARMONIC available at https://github.com/astro-informatics/harmonic

References in corpus (9)

• Bayes in the sky: Bayesian inference and model selection in cosmology
• PolyChord: nested sampling for cosmology
• Fast likelihood-free cosmology with neural density estimators and active learning
• Nested sampling for physical scientists
• Nested Sampling Methods
• Fast and Credible Likelihood-Free Cosmology with Truncated Marginal Neural Ratio Estimation
• Robust sampling for weak lensing and clustering analyses with the Dark Energy Survey
• SICRET: Supernova Ia Cosmology with truncated marginal neural Ratio EsTimation
• Proximal nested sampling for high-dimensional Bayesian model selection

Cited by in corpus (8)

• Impacts of dark energy on weighing neutrinos after DESI BAO
• KiDS-SBI: Simulation-based inference analysis of KiDS-1000 cosmic shear
• Evidence Networks: simple losses for fast, amortized, neural Bayesian model comparison
• Bayesian evidence estimation from posterior samples with normalizing flows
• Simulation-Based Inference of the sky-averaged 21-cm signal from CD-EoR with REACH
• Radio pulsar population synthesis with consistent flux measurements using simulation-based inference
• Deep inference of simulated strong lenses in ground-based surveys
• Flexible Simulation Based Inference for Galaxy Photometric Fitting with Synthesizer