Efficient neutral-IGM inference from noisy 21-cm forest spectra with latent-space U-Net encoding and XGBoost
arXiv:2507.11611 · doi:10.1093/mnras/stag236
Abstract
The 21-cm forest, comprising narrow absorption features imprinted on the radio spectra of high-redshift radio-loud quasars by intervening neutral hydrogen, offers a uniquely sensitive probe of the thermal state of the neutral intergalactic medium (IGM) during the epoch of reionization. Although over 30 such quasars are now known at , the signal remains elusive in practice, owing to instrumental noise, the intrinsic weakness of the absorption features, and the limited brightness of available background sources. Recent studies have focused on the one-dimensional transmission power spectrum as a statistical observable, but this approach also demands high signal-to-noise ratios. Here, we present a systematic comparison of five inference pipelines for recovering IGM parameters from mock 21-cm forest spectra at , incorporating realistic instrumental noise and telescope characteristics. We show that likelihood-free inference based on machine learning substantially outperforms traditional Bayesian methods. In particular, our most effective method dispenses with the power spectrum entirely: we use a convolutional U-Net to extract a latent-space encoding of the input spectrum and perform parameter regression using XGBoost. This approach yields accurate constraints on the IGM neutral fraction and X-ray heating efficiency even with a single 50-hour uGMRT sightline, which is an orders-of-magnitude improvement in integration time relative to existing techniques. We publicly release our code, training data, and models. Beyond the 21-cm forest, these results underscore the promise of hybrid deep learning and gradient-boosted inference techniques for extracting physical information from low-SNR data across astrophysics.
16 pages, 10 figures
References in corpus (70)
- Scikit-learn: Machine Learning in Python
- XGBoost: A Scalable Tree Boosting System
- SciPy 1.0--Fundamental Algorithms for Scientific Computing in Python
- Array Programming with NumPy
- Astropy: A Community Python Package for Astronomy
- emcee: The MCMC Hammer
- Planck 2013 results. XVI. Cosmological parameters
- Cosmology at Low Frequencies: The 21 cm Transition and the High-Redshift Universe
- 21cmFAST: A Fast, Semi-Numerical Simulation of the High-Redshift 21-cm Signal
- The LOFAR Two-metre Sky Survey - I. Survey Description and Preliminary Data Release
- Model-independent evidence in favor of an end to reionization by z~6
- Reionization and the Cosmic Dawn with the Square Kilometre Array
- GLEAM: The GaLactic and Extragalactic All-sky MWA survey
- Hydrogen reionisation ends by : Lyman- optical depth measured by the XQR-30 sample
- The Global 21 Centimeter Background from High Redshifts
- Large Lyman-alpha opacity fluctuations and low CMB tau in models of late reionization with large islands of neutral hydrogen extending to z<5.5
- Improved Constraints on the 21 cm EoR Power Spectrum and the X-Ray Heating of the IGM with HERA Phase I Observations
- The mean free path of ionizing photons at 5 < z < 6: evidence for rapid evolution near reionization
- HI 21cm absorption beyond the epoch of re-ionization
- 21cmFAST v3: A Python-integrated C code forgenerating 3D realizations of the cosmic 21cm signal
- The 21 cm Forest: Radio Absorption Spectra as Probes of Minihalos Before Reionization
- The LOFAR Two Metre Sky Survey: Deep Fields Data Release 1 -- III. Host-galaxy identifications and value added catalogues
- Deep learning from 21-cm tomography of the Cosmic Dawn and Reionization
- Anticipated Performance of the Square Kilometre Array -- Phase 1 (SKA1)
- The PHLEK Survey: A New Determination of the Primordial Helium Abundance
- The first blazar observed at z>6
- Long Dark Gaps in the Ly Forest at : Evidence of Ultra Late Reionization from XQR-30 Spectra
- Deep Learning of Quasar Spectra to Discover and Characterize Damped Lya Systems
- Interpreting LOFAR 21-cm signal upper limits at z~9.1 in the context of high-z galaxy and reionisation observations
- Exploring reionisation and high-z galaxy observables with recent multi-redshift MWA upper limits on the 21-cm signal
- deep21: a Deep Learning Method for 21cm Foreground Removal
- Learning to Denoise Astronomical Images with U-nets
- Photometry of high-redshift blended galaxies using deep learning
- Chronicling the reionization history at with emergent quasar damping wings
- Probing small-scale cosmological fluctuations with the 21 cm forest: effects of neutrino mass, running spectral index and warm dark matter
- Detecting ionized bubbles in redshifted 21 cm maps
- Prediction of galaxy halo masses in SDSS DR7 via a machine learning approach
- Photometric redshift-aided classification using ensemble learning
- Initial results from a realtime FRB search with the GBT
- The 21 cm Forest as a Probe of the Reionization and the Temperature of the Intergalactic Medium
- Prospects for detecting the 21cm forest from the diffuse intergalactic medium with LOFAR
- Detecting the redshifted 21cm forest during reionization
- The earliest galaxies seen in 21 cm line absorption
- SHAPing the Gas: Understanding Gas Shapes in Dark Matter Haloes with Interpretable Machine Learning
- Constraints on nature of ultra light dark matter particles with 21cm forest
- Detecting the 21 cm Forest in the 21 cm Power Spectrum
- Debiasing inference with approximate covariance matrices and other unidentified biases
- The 21-cm forest as a simultaneous probe of dark matter and cosmic heating history
- The impact of non-Gaussianity on the Epoch of Reionization parameter forecast using 21-cm power spectrum measurements
- Exploiting synergies between JWST and cosmic 21-cm observations to uncover star formation in the early Universe
- Simulating the 21cm forest detectable with LOFAR and SKA in the spectra of high-z GRBs
- The detectability of strong 21 centimetre forest absorbers from the diffuse intergalactic medium in late reionisation models
- Probing Reionization and Early Cosmic Enrichment with the MgII Forest
- Evolution of cold gas at 2<z<5: a blind search for HI and OH absorption lines towards mid-infrared color selected radio-loud AGNs
- Statistical Detection of IGM Structures during Cosmic Reionization using Absorption of the Redshifted 21 cm line by HI against Compact Background Radio Sources
- 21cm forest probes on the axion dark matter in the post-inflationary Peccei-Quinn symmetry breaking scenarios
- Deep Forest: Neural Network reconstruction of the Lyman-alpha forest
- Probing quasar lifetimes with proximate -centimetre absorption in the diffuse intergalactic medium at redshifts
- From Halos to Galaxies. VI. Improved halo mass estimation for SDSS groups and measurement of the halo mass function
- Deep learning-driven likelihood-free parameter inference for 21-cm forest observations
- SNGuess: A method for the selection of young extragalactic transients
- Particle initialization effects on Lyman- forest statistics in cosmological SPH simulations
- Prospects for Observing High-redshift Radio-loud Quasars in the SKA Era: Paving the Way for 21-cm Forest Observations
- Analyzing the 21cm forest with Wavelet Scattering Transform: Insight into non-Gaussian features of the 21cm forest
- Prospects of a statistical detection of the 21-cm forest and its potential to constrain the thermal state of the neutral IGM during reionization
- Impact of dark matter-baryon relative velocity on the 21cm forest
- Forecasting constraints on the high-z IGM thermal state from the Lyman- forest flux auto-correlation function
- Prospects for measuring neutrino mass with 21-cm forest
- Deblending Overlapping Galaxies in DECaLS Using Transformer-Based Algorithm: A Method Combining Multiple Bands and Data Types
- Analytical modeling of the one-dimensional power spectrum of 21-cm forest based on a halo model method