A general Bayesian model-validation framework based on null-test evidence ratios, with an example application to global 21-cm cosmology

Abstract

Comparing composite models for multi-component observational data is a prevalent scientific challenge. When fitting composite models, there exists the potential for systematics from a poor fit of one model component to be absorbed by another, resulting in the composite model providing an accurate fit to the data in aggregate but yielding biased a posteriori estimates for individual components. We begin by defining a classification scheme for composite model comparison scenarios, identifying two categories: category I, where models with accurate and predictive components are separable through Bayesian comparison of the unvalidated composite models, and category II, where models with accurate and predictive components may not be separable due to interactions between components, leading to spurious detections or biased signal estimation. To address the limitations of category II model comparisons, we introduce the Bayesian Null Test Evidence Ratio-based (BaNTER) validation framework. Applying this classification scheme and BaNTER to a composite model comparison problem in 21-cm cosmology, where minor systematics from imperfect foreground modelling can bias global 21-cm signal recovery, we validate six composite models using mock data. We show that incorporating BaNTER alongside Bayes-factor-based comparison reliably ensures unbiased inferences of the signal of interest across both categories, positioning BaNTER as a valuable addition to Bayesian inference workflows with potential applications across diverse fields.

20 pages, 5 figures; accepted for publication in MNRAS

References in corpus (12)

• Bayes in the sky: Bayesian inference and model selection in cosmology
• PolyChord: nested sampling for cosmology
• High Tide or Riptide on the Cosmic Shoreline? A Water-Rich Atmosphere or Stellar Contamination for the Warm Super-Earth GJ~486b from JWST Observations
• Testing for calibration systematics in the EDGES low-band data using Bayesian model selection
• Contamination of the Epoch of Reionization power spectrum in the presence of foregrounds
• Modelling the Galactic Foreground and Beam Chromaticity for Global 21-cm Cosmology
• A Bayesian approach to modelling spectrometer data chromaticity corrected using beam factors -- I. Mathematical formalism
• A General Bayesian Framework to Account for Foreground Map Errors in Global 21-cm Experiments
• Sky-averaged 21-cm signal extraction using multiple antennas with an SVD framework: the REACH case
• A Bayesian approach to high fidelity interferometric calibration II: demonstration with simulated data
• All Sky Modelling Requirements for Bayesian 21 cm Power Spectrum Estimation with BayesEoR
• Simulation-Based Inference of the sky-averaged 21-cm signal from CD-EoR with REACH

Cited by in corpus (3)

• Rapid and Late Cosmic Reionization Driven by Massive Galaxies: a Joint Analysis of Constraints from 21-cm, Lyman Line & CMB Data Sets
• A Bayesian approach to modelling spectrometer data chromaticity corrected using beam factors -- II. Model priors and posterior odds
• Optimizing Foreground Modelling for Global 21cm Cosmology with GPU-Accelerated Nested Sampling