A Differentiable, End-to-End Forward Model for 21 cm Cosmology: Estimating the Foreground, Instrument, and Signal Joint Posterior
arXiv:2504.07090 · doi:10.1093/mnras/staf1007
Abstract
We present a differentiable, end-to-end Bayesian forward modeling framework for line intensity mapping cosmology experiments, with a specific focus on low-frequency radio telescopes targeting the redshifted 21 cm line from neutral hydrogen as a cosmological probe. Our framework is capable of posterior density estimation of the cosmological signal jointly with foreground and telescope parameters at the field level. Our key aim is to be able to optimize the model's high-dimensional, non-linear, and ill-conditioned parameter space, while also sampling from it to perform robust uncertainty quantification within a Bayesian framework. We show how a differentiable programming paradigm, accelerated by recent advances in machine learning software and hardware, can make this computationally-demanding, end-to-end Bayesian approach feasible. We demonstrate a proof-of-concept on a signal recovery problem for the Hydrogen Epoch of Reionization Array experiment, highlighting the framework's ability to build confidence in early 21 cm signal detections even in the presence of poorly understood foregrounds and instrumental systematics. We use a Hessian-preconditioned Hamiltonian Monte Carlo algorithm to efficiently sample our parameter space with a dimensionality approaching , which enables joint, end-to-end nuisance parameter marginalization over foreground and instrumental terms. Lastly, we introduce a new spherical harmonic formalism that is a complete and orthogonal basis on the cut sky relevant to drift-scan radio surveys, which we call the spherical stripe harmonic formalism, and it's associated three-dimensional basis, the spherical stripe Fourier-Bessel formalism.
27 pages, 15 figures; Accepted to MNRAS
References in corpus (141)
- HEALPix -- a Framework for High Resolution Discretization, and Fast Analysis of Data Distributed on the Sphere
- MCMC using Hamiltonian dynamics
- The No-U-Turn Sampler: Adaptively Setting Path Lengths in Hamiltonian Monte Carlo
- Rank-normalization, folding, and localization: An improved for assessing convergence of MCMC
- Cosmology at Low Frequencies: The 21 cm Transition and the High-Redshift Universe
- Dark Energy Survey Year 3 Results: Cosmological Constraints from Galaxy Clustering and Weak Lensing
- An absorption profile centred at 78 megahertz in the sky-averaged spectrum
- 21-cm cosmology
- Cosmic Reionization and Early Star-Forming Galaxies: A Joint Analysis of New Constraints from Planck and Hubble Space Telescope
- 21cmFAST: A Fast, Semi-Numerical Simulation of the High-Redshift 21-cm Signal
- Hydrogen Epoch of Reionization Array (HERA)
- GaLactic and Extragalactic All-sky Murchison Widefield Array (GLEAM) survey I: A low-frequency extragalactic catalogue
- A model of diffuse Galactic Radio Emission from 10 MHz to 100 GHz
- Reionization and Cosmology with 21 cm Fluctuations
- The VLA Low-frequency Sky Survey
- The Universe is Reionizing at z~7: Bayesian Inference of the IGM Neutral Fraction Using Ly Emission from Galaxies
- Measuring the Small-Scale Power Spectrum of Cosmic Density Fluctuations Through 21 cm Tomography Prior to the Epoch of Structure Formation
- Bright Source Subtraction Requirements For Redshifted 21 cm Measurements
- What Next-Generation 21 cm Power Spectrum Measurements Can Teach Us About the Epoch of Reionization
- Measurement of 21 cm brightness fluctuations at z ~ 0.8 in cross-correlation
- Late-time cosmology with 21cm intensity mapping experiments
- A simulation calibrated limit on the HI power spectrum from the GMRT Epoch of Reionization experiment
- A Per-Baseline, Delay-Spectrum Technique for Accessing the 21cm Cosmic Reionization Signature
- Bayesian physical reconstruction of initial conditions from large scale structure surveys
- Four Fundamental Foreground Power Spectrum Shapes for 21 cm Cosmology Observations
- Improved upper limits on the 21-cm signal power spectrum of neutral hydrogen at from LOFAR
- Joint Bayesian component separation and CMB power spectrum estimation
- How accurately can 21 cm tomography constrain cosmology?
- The Epoch of Reionization Window: I. Mathematical Formalism
- Identification and properties of intense star-forming galaxies at redshifts z>10
- Revisiting the radio interferometer measurement equation. I. A full-sky Jones formalism
- Inferring the astrophysics of reionization and cosmic dawn from galaxy luminosity functions and the 21-cm signal
- Toward Epoch of Reionization Measurements with Wide-Field Radio Observations
- Data Analysis for Precision 21 cm Cosmology
- Imaging the Epoch of Reionization: limitations from foreground confusion and imaging algorithms
- Deep multi-redshift limits on Epoch of Reionisation 21cm Power Spectra from Four Seasons of Murchison Widefield Array Observations
- Global, Exact Cosmic Microwave Background Data Analysis Using Gibbs Sampling
- Power spectrum estimation from high-resolution maps by Gibbs sampling
- The impact of point source subtraction residuals on 21 cm Epoch of Reionization estimation
- The observable signature of late heating of the Universe during cosmic reionization
- First Results from HERA Phase I: Upper Limits on the Epoch of Reionization 21 cm Power Spectrum
- An Improved Model of Diffuse Galactic Radio Emission from 10 MHz to 5 THz
- Overcoming real-world obstacles in 21 cm power spectrum estimation: A method demonstration and results from early Murchison Widefield Array data
- All-Sky Interferometry with Spherical Harmonic Transit Telescopes
- Improved Constraints on the 21 cm EoR Power Spectrum and the X-Ray Heating of the IGM with HERA Phase I Observations
- A Method for 21cm Power Spectrum Estimation in the Presence of Foregrounds
- Opening the 21cm EoR Window: Measurements of Foreground Isolation with PAPER
- The power spectrum of galaxies in the 2dF 100k redshift survey
- HERA Phase I Limits on the Cosmic 21-cm Signal: Constraints on Astrophysics and Cosmology During the Epoch of Reionization
- Foregrounds in Wide-Field Redshifted 21 cm Power Spectra
- Testing for calibration systematics in the EDGES low-band data using Bayesian model selection
- A simplified, lossless re-analysis of PAPER-64
- Application of Monte Carlo Algorithms to the Bayesian Analysis of the Cosmic Microwave Background
- Realistic Simulations of the Galactic Polarized Foreground: Consequences for 21-cm Reionization Detection Experiments
- Hyper-efficient model-independent Bayesian method for the analysis of pulsar timing data
- Statistical 21-cm Signal Separation via Gaussian Process Regression Analysis
- Bayesian constraints on the global 21-cm signal from the Cosmic Dawn
- Are the ultra-high-redshift galaxies at z > 10 surprising in the context of standard galaxy formation models?
- Fast Holographic Deconvolution: a new technique for precision radio interferometry
- SARAS 2 constraints on global 21-cm signals from the Epoch of Reionization
- The kinetic Sunyaev-Zel'dovich signal from inhomogeneous reionization: a parameter space study
- The LWA1 Low Frequency Sky Survey
- Confirmation of Wide-Field Signatures in Redshifted 21 cm Power Spectra
- The Importance of Wide-field Foreground Removal for 21 cm Cosmology: A Demonstration With Early MWA Epoch of Reionization Observations
- Detection of Cosmological 21 cm Emission with the Canadian Hydrogen Intensity Mapping Experiment
- The Radio Sky at Meter Wavelengths: m-Mode Analysis Imaging with the Owens Valley Long Wavelength Array
- Reionization after JWST: a photon budget crisis?
- A Scalable Correlator Architecture Based on Modular FPGA Hardware, Reuseable Gateware, and Data Packetization
- The Effects of Polarized Foregrounds on 21cm Epoch of Reionization Power Spectrum Measurements
- Mitigating Internal Instrument Coupling for 21 cm Cosmology I: Temporal and Spectral Modeling in Simulations
- Mitigating Internal Instrument Coupling II: A Method Demonstration with the Hydrogen Epoch of Reionization Array
- A General Bayesian Framework for Foreground Modelling and Chromaticity Correction for Global 21cm Experiments
- High-redshift post-reionization cosmology with 21cm intensity mapping
- Understanding the Diversity of 21 cm Cosmology Analyses
- Calibration and Stokes Imaging with Full Embedded Element Primary Beam Model for the Murchison Widefield Array
- Spectral Index of the Diffuse Radio Background Between 50 and 100 MHz
- A generalised Measurement Equation and van Cittert-Zernike theorem for wide-field radio astronomical interferometry
- First upper limits on the 21 cm signal power spectrum from cosmic dawn from one night of observations with NenuFAR
- Comparing Foreground Removal Techniques for Recovery of the LOFAR-EoR 21cm Power Spectrum
- Gaussian Process Foreground Subtraction and Power Spectrum Estimation for 21 cm Cosmology
- JAX-COSMO: An End-to-End Differentiable and GPU Accelerated Cosmology Library
- How much information can be extracted from galaxy clustering at the field level?
- Spherical Harmonic Analyses of Intensity Mapping Power Spectra
- Modelling of the Spectral Energy Distribution of Fornax A: Leptonic and Hadronic Production of High Energy Emission from the Radio Lobes
- The FHD/ppsilon Epoch of Reionization Power Spectrum Pipeline
- Mapmaking for Precision 21 cm Cosmology
- DAYENU: A Simple Filter of Smooth Foregrounds for Intensity Mapping Power Spectra
- Absolute Calibration Strategies for the Hydrogen Epoch of Reionization Array and Their Impact on the 21 cm Power Spectrum
- Global 21-cm signal extraction from foreground and instrumental effects I: Pattern recognition framework for separation using training sets
- 3DEX: a code for fast spherical Fourier-Bessel decomposition of 3D surveys
- Optimized Beam Sculpting with Generalized Fringe-Rate Filters
- In situ measurement of MWA primary beam variation using ORBCOMM
- 3D Spherical Analysis of Baryon Acoustic Oscillations
- Global 21-cm Signal Extraction from Foreground and Instrumental Effects II: Efficient and Self-Consistent Technique for Constraining Nonlinear Signal Models
- A Unified Calibration Framework for 21 cm Cosmology
- 21cmEMU: an emulator of 21cmFAST summary observables
- The POlarised GLEAM Survey (POGS) II: Results from an All-Sky Rotation Measure Synthesis Survey at Long Wavelengths
- GIGA-Lens: Fast Bayesian Inference for Strong Gravitational Lens Modeling
- Bayesian semi-blind component separation for foreground removal in interferometric 21-cm observations
- Hydrogen Epoch of Reionization Array (HERA) Phase II Deployment and Commissioning
- Constraining Polarized Foregrounds for EoR Experiments I: 2D Power Spectra from the PAPER-32 Imaging Array
- Forward modeling of galaxy populations for cosmological redshift distribution inference
- Use of Time Dependent Data in Bayesian Global 21cm Foreground and Signal Modelling
- Array Element Coupling in Radio Interferometry I: A Semi-Analytic Approach
- Constraining Polarized Foregrounds for EOR Experiments II: Polarization Leakage Simulations in the Avoidance Scheme
- Lost Horizon: Quantifying the Effect of Local Topography on Global 21-cm Cosmology Data Analysis
- New EoR Power Spectrum Limits From MWA Phase II Using the Delay Spectrum Method and Novel Systematic Rejection
- BeyondPlanck I. Global Bayesian analysis of the Planck Low Frequency Instrument data
- Validation of the HERA Phase I Epoch of Reionization 21 cm Power Spectrum Software Pipeline
- A Bayesian Calibration Framework for EDGES
- Spectral index of the Galactic foreground emission in the 50-87 MHz range
- Design of the New Wideband Vivaldi Feed for the HERA Radio-Telescope Phase II
- A Map of Diffuse Radio Emission at 182 MHz to Enhance Epoch of Reionization Observations in the Southern Hemisphere
- SuperFaB: a fabulous code for Spherical Fourier-Bessel decomposition
- A direct detection of neutral hydrogen intensity mapping on Mpc scales at and
- TARDIS Paper II: Synergistic Density Reconstruction from Lyman-alpha Forest and Spectroscopic Galaxy Surveys with Applications to Protoclusters and the Cosmic Web
- Modelling and peeling extended sources with shapelets: a Fornax A case study
- Effects of Linear Redshift Space Distortions and Perturbation Theory on BAOs: A 3D Spherical Analysis
- Measuring HERA's primary beam in-situ: methodology and first results
- Statistical recovery of 21cm visibilities and their power spectra with Gaussian constrained realisations and Gibbs sampling
- How informative are summaries of the cosmic 21-cm signal?
- Differentiable Cosmological Simulation with Adjoint Method
- Methods of Error Estimation for Delay Power Spectra in Cosmology
- Quantifying EoR delay spectrum contamination from diffuse radio emission
- A General Bayesian Framework to Account for Foreground Map Errors in Global 21-cm Experiments
- On the use of temporal filtering for mitigating galactic synchrotron calibration bias in 21 cm reionization observations
- Forecasts and Statistical Insights for Line Intensity Mapping Cross-Correlations: A Case Study with 21cm x [CII]
- All Sky Modelling Requirements for Bayesian 21 cm Power Spectrum Estimation with BayesEoR
- Bayesian evidence-driven likelihood selection for sky-averaged 21-cm signal extraction
- Higher Order Hamiltonian Monte Carlo Sampling for Cosmological Large-Scale Structure Analysis
- Sensitivity of Bayesian 21 cm power spectrum estimation to foreground model errors
- Bayesian Multi-line Intensity Mapping
- Full-Sky Lensing Reconstruction of 21 cm Intensity Maps
- Measurements of one-point statistics in 21 cm intensity maps via foreground avoidance strategy
- Statistical estimation of full-sky radio maps from 21cm array visibility data using Gaussian constrained realizations
- Investigating Mutual Coupling in the Hydrogen Epoch of Reionization Array and Mitigating its Effects on the 21-cm Power Spectrum
- Bayesian jackknife tests with a small number of subsets: Application to HERA 21cm power spectrum upper limits
- Bayesian estimation of cross-coupling and reflection systematics in 21cm array visibility data
- Delayed rejection Hamiltonian Monte Carlo for sampling multiscale distributions
- Limits on the 21 cm power spectrum at z=6.5-7.0 from MWA observations
- Neural Posterior Estimation with Differentiable Simulators
Cited by in corpus (4)
- Field-level Reconstruction from Foreground-Contaminated 21-cm Maps
- A Bayesian approach to modelling spectrometer data chromaticity corrected using beam factors -- II. Model priors and posterior odds
- Impact of 21-cm foreground mitigation strategies on reionization power spectrum constraints
- Optimizing Foreground Modelling for Global 21cm Cosmology with GPU-Accelerated Nested Sampling