A general Bayesian model-validation framework based on null-test evidence ratios, with an example application to global 21-cm cosmology

Abstract

Comparing composite models for multi-component observational data is a prevalent scientific challenge. When fitting composite models, there exists the potential for systematics from a poor fit of one model component to be absorbed by another, resulting in the composite model providing an accurate fit to the data in aggregate but yielding biased a posteriori estimates for individual components. We begin by defining a classification scheme for composite model comparison scenarios, identifying two categories: category I, where models with accurate and predictive components are separable through Bayesian comparison of the unvalidated composite models, and category II, where models with accurate and predictive components may not be separable due to interactions between components, leading to spurious detections or biased signal estimation. To address the limitations of category II model comparisons, we introduce the Bayesian Null Test Evidence Ratio-based (BaNTER) validation framework. Applying this classification scheme and BaNTER to a composite model comparison problem in 21-cm cosmology, where minor systematics from imperfect foreground modelling can bias global 21-cm signal recovery, we validate six composite models using mock data. We show that incorporating BaNTER alongside Bayes-factor-based comparison reliably ensures unbiased inferences of the signal of interest across both categories, positioning BaNTER as a valuable addition to Bayesian inference workflows with potential applications across diverse fields.

20 pages, 5 figures; accepted for publication in MNRAS

References in corpus (26)

• Bayes in the sky: Bayesian inference and model selection in cosmology
• An absorption profile centred at 78 megahertz in the sky-averaged spectrum
• PolyChord: nested sampling for cosmology
• PolyChord: next-generation nested sampling
• Concerns about Modelling of the EDGES Data
• Data Analysis for Precision 21 cm Cosmology
• High Tide or Riptide on the Cosmic Shoreline? A Water-Rich Atmosphere or Stellar Contamination for the Warm Super-Earth GJ~486b from JWST Observations
• Testing for calibration systematics in the EDGES low-band data using Bayesian model selection
• A Ground Plane Artifact that Induces an Absorption Profile in Averaged Spectra from Global 21-cm Measurements - with Possible Application to EDGES
• The redshifted 21-cm signal in the EDGES low-band spectrum
• A General Bayesian Framework for Foreground Modelling and Chromaticity Correction for Global 21cm Experiments
• fgivenx: A Python package for functional posterior plotting
• maxsmooth: Rapid Maximally Smooth Function Fitting With Applications in Global 21-cm Cosmology
• Global 21-cm signal extraction from foreground and instrumental effects I: Pattern recognition framework for separation using training sets
• On the contamination of the global 21~cm signal from polarized foregrounds
• Global 21-cm Signal Extraction from Foreground and Instrumental Effects II: Efficient and Self-Consistent Technique for Constraining Nonlinear Signal Models
• Contamination of the Epoch of Reionization power spectrum in the presence of foregrounds
• Joint estimation of the Epoch of Reionization power spectrum and foregrounds
• Modelling the Galactic Foreground and Beam Chromaticity for Global 21-cm Cosmology
• A Bayesian approach to modelling spectrometer data chromaticity corrected using beam factors -- I. Mathematical formalism
• A General Bayesian Framework to Account for Foreground Map Errors in Global 21-cm Experiments
• Sky-averaged 21-cm signal extraction using multiple antennas with an SVD framework: the REACH case
• A Bayesian approach to high fidelity interferometric calibration II: demonstration with simulated data
• Bayesian Data Analysis for Sky-averaged 21-cm Experiments in the Presence of Ionospheric Effects
• All Sky Modelling Requirements for Bayesian 21 cm Power Spectrum Estimation with BayesEoR
• Simulation-Based Inference of the sky-averaged 21-cm signal from CD-EoR with REACH

Cited by in corpus (3)

• Rapid and Late Cosmic Reionization Driven by Massive Galaxies: a Joint Analysis of Constraints from 21-cm, Lyman Line & CMB Data Sets
• A Bayesian approach to modelling spectrometer data chromaticity corrected using beam factors -- II. Model priors and posterior odds
• Optimizing Foreground Modelling for Global 21cm Cosmology with GPU-Accelerated Nested Sampling