Evidence Networks: simple losses for fast, amortized, neural Bayesian model comparison

Abstract

Evidence Networks can enable Bayesian model comparison when state-of-the-art methods (e.g. nested sampling) fail and even when likelihoods or priors are intractable or unknown. Bayesian model comparison, i.e. the computation of Bayes factors or evidence ratios, can be cast as an optimization problem. Though the Bayesian interpretation of optimal classification is well-known, here we change perspective and present classes of loss functions that result in fast, amortized neural estimators that directly estimate convenient functions of the Bayes factor. This mitigates numerical inaccuracies associated with estimating individual model probabilities. We introduce the leaky parity-odd power (l-POP) transform, leading to the novel ``l-POP-Exponential'' loss function. We explore neural density estimation for data probability in different models, showing it to be less accurate and scalable than Evidence Networks. Multiple real-world and synthetic examples illustrate that Evidence Networks are explicitly independent of dimensionality of the parameter space and scale mildly with the complexity of the posterior probability density function. This simple yet powerful approach has broad implications for model inference tasks. As an application of Evidence Networks to real-world data we compute the Bayes factor for two models with gravitational lensing data of the Dark Energy Survey. We briefly discuss applications of our methods to other, related problems of model comparison and evaluation in implicit inference settings.

arXiv comment21 pages, 8 figures, accepted by Machine Learning: Science and Technology

References in corpus (22)

• Adam: A Method for Stochastic Optimization
• Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift
• MultiNest: an efficient and robust Bayesian inference tool for cosmology and particle physics
• Empirical Evaluation of Rectified Activations in Convolutional Network
• PolyChord: next-generation nested sampling
• Dark Energy Survey Year 3 Results: Cosmology from Cosmic Shear and Robustness to Data Calibration
• Dark Energy Survey Year 3 Results: Cosmology from Cosmic Shear and Robustness to Modeling Uncertainty
• Fast likelihood-free cosmology with neural density estimators and active learning
• Mining gold from implicit models to improve likelihood-free inference
• Massive optimal data compression and density estimation for scalable, likelihood-free inference in cosmology
• Obtaining Calibrated Probabilities from Boosting
• Likelihood-free inference with neural compression of DES SV weak lensing map statistics
• Dark Energy Survey Year 3 results: curved-sky weak lensing mass map reconstruction
• Bayesian Evidence and Model Selection
• Mining for Dark Matter Substructure: Inferring subhalo population properties from strong lenses with machine learning
• Dark Energy Survey Year 3 results: cosmological constraints from the analysis of cosmic shear in harmonic space
• Bayesian inflationary reconstructions from Planck 2018 data
• Dynamical mass inference of galaxy clusters with neural flows
• Cosmic Shear: Inference from Forward Models
• The sum of the masses of the Milky Way and M31: a likelihood-free inference approach
• Learning Likelihood Ratios with Neural Network Classifiers
• Bayesian model comparison for simulation-based inference

Cited by in corpus (7)

• Learning Likelihood Ratios with Neural Network Classifiers
• Bayesian evidence estimation from posterior samples with normalizing flows
• Enhancing Cosmological Model Selection with Interpretable Machine Learning
• Massive $\nu$s through the CNN lens: interpreting the field-level neutrino mass information in weak lensing
• Statistical Patterns in the Equations of Physics and the Emergence of a Meta-Law of Nature
• Flexible Simulation Based Inference for Galaxy Photometric Fitting with Synthesizer
• Optimizing Foreground Modelling for Global 21cm Cosmology with GPU-Accelerated Nested Sampling