Machine learning astrophysics from 21 cm lightcones: impact of network architectures and signal contamination

Abstract

Imaging the cosmic 21 cm signal will map out the first billion years of our Universe. The resulting 3D lightcone (LC) will encode the properties of the unseen first galaxies and physical cosmology. Here, we build on previous work using neural networks (NNs) to infer astrophysical parameters directly from 21 cm LC images. We introduce recurrent neural networks (RNNs), capable of efficiently characterizing the evolution along the redshift axis of 21 cm LC images. Using a large database of simulated cosmic 21 cm LCs, we compare the relative performance in parameter estimation of different network architectures. These including two types of RNNs, which differ in their complexity, as well as a more traditional convolutional neural network (CNN). For the ideal case of no instrumental effects, our simplest and easiest to train RNN performs the best, with a mean squared parameter estimation error (MSE) that is lower by a factor of $\ge 2$ compared with the other architectures studied here, and a factor of $\ge 8$ lower than the previously-studied CNN. We also corrupt the cosmic signal by adding noise expected from a 1000 h integration with the Square Kilometre Array, as well as excising a foreground-contaminated 'horizon wedge'. Parameter prediction errors increase when the NNs are trained on these contaminated LC images, though recovery is still good even in the most pessimistic case (with $R^2\ge 0.5-0.95$). However, we find no notable differences in performance between network architectures on the contaminated images. We argue this is due to the size of our data set, highlighting the need for larger data sets and/or better data augmentation in order to maximize the potential of NNs in 21 cm parameter estimation.

arXiv comment15 pages, 11 figures, updated to match the version published in MNRAS, minor changes

References in corpus (63)

• Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift
• Convolutional LSTM Network: A Machine Learning Approach for Precipitation Nowcasting
• WaveNet: A Generative Model for Raw Audio
• The Effectiveness of Data Augmentation in Image Classification using Deep Learning
• Accurate, Large Minibatch SGD: Training ImageNet in 1 Hour
• Cosmology at Low Frequencies: The 21 cm Transition and the High-Redshift Universe
• A guide to convolution arithmetic for deep learning
• Unmasking Clever Hans Predictors and Assessing What Machines Really Learn
• Delving Deep into Rectifiers: Surpassing Human-Level Performance on ImageNet Classification
• Visualizing and Understanding Recurrent Networks
• 21cmFAST: A Fast, Semi-Numerical Simulation of the High-Redshift 21-cm Signal
• The Growth of HII Regions During Reionization
• Efficient Simulations of Early Structure Formation and Reionization
• What Next-Generation 21 cm Power Spectrum Measurements Can Teach Us About the Epoch of Reionization
• Four Fundamental Foreground Power Spectrum Shapes for 21 cm Cosmology Observations
• Improved upper limits on the 21-cm signal power spectrum of neutral hydrogen at $\mathbf{z}\approx 9.1$ from LOFAR
• Unusually Large Fluctuations in the Statistics of Galaxy Formation at High Redshift
• The Epoch of Reionization Window: I. Mathematical Formalism
• New Limits on 21cm EoR From PAPER-32 Consistent with an X-Ray Heated IGM at z=7.7
• Inferring the astrophysics of reionization and cosmic dawn from galaxy luminosity functions and the 21-cm signal
• Imaging the Epoch of Reionization: limitations from foreground confusion and imaging algorithms
• The Epoch of Reionization Window: II. Statistical Methods for Foreground Wedge Reduction
• The impact of point source subtraction residuals on 21 cm Epoch of Reionization estimation
• A Sensitivity and Array-Configuration Study for Measuring the Power Spectrum of 21cm Emission from Reionization
• Inhomogeneous recombinations during cosmic reionization
• Recurrent Neural Networks (RNNs): A gentle Introduction and Overview
• The Global 21-cm Signal in the Context of the High-z Galaxy Luminosity Function
• Improving the Epoch of Reionization Power Spectrum Results from Murchison Widefield Array Season 1 Observations
• CHIPS: The Cosmological HI Power Spectrum Estimator
• Quantifying the non-Gaussianity in the EoR 21-cm signal through bispectrum
• Emulating Simulations of Cosmic Dawn for 21cm Power Spectrum Constraints on Cosmology, Reionization, and X-ray Heating
• Constraining the intergalactic medium at $z\approx$ 9.1 using LOFAR Epoch of Reionization observations
• Deep learning from 21-cm tomography of the Cosmic Dawn and Reionization
• 21CMMC with a 3D light-cone: the impact of the co-evolution approximation on the astrophysics of reionisation and cosmic dawn
• Emulation of reionization simulations for Bayesian inference of astrophysics parameters using neural networks
• Reionization and Beyond: detecting the peaks of the cosmological 21cm signal
• Tight Constraints on the Excess Radio Background at $z=9.1$ from LOFAR
• Interpreting LOFAR 21-cm signal upper limits at z~9.1 in the context of high-z galaxy and reionisation observations
• Bubble size statistics during reionization from 21-cm tomography
• Constraining the astrophysics and cosmology from 21cm tomography using deep learning with the SKA
• Optimal identification of HII regions during reionization in 21-cm observations
• The 21cm bispectrum as a probe of non-Gaussianities due to X-ray heating
• deep21: a Deep Learning Method for 21cm Foreground Removal
• Comparing Foreground Removal Techniques for Recovery of the LOFAR-EoR 21cm Power Spectrum
• Constraining the EoR model parameters with the 21cm bispectrum
• Comparing Redundant and Sky Model Based Interferometric Calibration: A First Look with Phase II of the MWA
• Evaluating machine learning techniques for predicting power spectra from reionization simulations
• Recovering the Wedge Modes Lost to 21-cm Foregrounds
• The visibility based Tapered Gridded Estimator (TGE) for the redshifted 21-cm power spectrum
• Studying 21cm power spectrum with one-point statistics
• Neutral island statistics during reionization from 21-cm tomography
• Improved 21 cm Epoch Of Reionization Power Spectrum Measurements with a Hybrid Foreground Subtraction and Avoidance Technique
• Deep learning approach for identification of HII regions during reionization in 21-cm observations
• Constraining the reionization history using deep learning from 21cm tomography with the Square Kilometre Array
• Machine Learning Applied to the Reionization History of the Universe in the 21 cm Signal
• Properties of reionization-era galaxies from JWST luminosity functions and 21-cm interferometry
• The effect of baseline layouts on the EoR foreground wedge: a semi-analytical approach
• Inferring the properties of the sources of reionization using the morphological spectra of the ionized regions
• Studying the morphology of reionisation with the triangle correlation function of phases
• Constraining the Reionization History using Bayesian Normalizing Flows
• Deep-Learning Study of the 21cm Differential Brightness Temperature During the Epoch of Reionization
• Hierarchical Recurrent Neural Network for Video Summarization
• Gradual Learning of Recurrent Neural Networks

Cited by in corpus (30)

• Implicit Likelihood Inference of Reionization Parameters from the 21 cm Power Spectrum
• Exploring the likelihood of the 21-cm power spectrum with simulation-based inference
• Exploring the cosmic 21-cm signal from the Epoch of Reionisation using the Wavelet Scattering Transform
• 21cmEMU: an emulator of 21cmFAST summary observables
• Machine Learning for Observational Cosmology
• HIFlow: Generating Diverse HI Maps and Inferring Cosmology while Marginalizing over Astrophysics using Normalizing Flows
• Deep learning approach for identification of HII regions during reionization in 21-cm observations -- II. foreground contamination
• Inferring Astrophysics and Dark Matter Properties from 21cm Tomography using Deep Learning
• Measuring the Hubble Constant with cosmic chronometers: a machine learning approach
• The importance of stochasticity in determining galaxy emissivities and UV LFs during cosmic dawn and reionization
• How informative are summaries of the cosmic 21-cm signal?
• Exploring the cosmic dawn and epoch of reionization with 21cm line
• Machine-learning recovery of foreground wedge-removed 21-cm light cones for high-$z$ galaxy mapping
• Detecting the non-Gaussianity of the 21-cm signal during reionisation with the Wavelet Scattering Transform
• Neural networks: solving the chemistry of the interstellar medium
• AI-driven spatio-temporal engine for finding gravitationally lensed type Ia supernovae
• Measurements of one-point statistics in 21 cm intensity maps via foreground avoidance strategy
• Predictive uncertainty on improved astrophysics recovery from multifield cosmology
• Reionisation time fields reconstruction from 21 cm signal maps
• Exploring One-point Statistics in HERA Phase I Data: Effects of Foregrounds and Systematics on Measuring One-Point Statistics
• 3D ScatterNet: Inference from 21 cm Light-cones
• From ANN to BNN: Inferring Reionization Parameters using Uncertainty-aware Emulators of 21-cm Summaries
• Imprints of fermionic and bosonic mixed dark matter on the 21-cm signal at cosmic dawn
• Accelerating reionization constraints: An ANN-emulator framework for the SCRIPT Semi-numerical Model
• Seeing Wiggles without Seeing Wiggles: BAO Recovery in 21 cm Intensity Mapping with Deep Learning
• Nonlinear reconstruction of 21cm global signal from 21cm power spectrum with artificial neural networks
• Beyond the Power Spectrum: A New Framework for Non-Stationary Fields with Applications to Light-Cone Effects in Line Intensity Mapping
• The averaging problem on the past null cone in inhomogeneous dust cosmologies
• A bubble size distribution model for the Epoch of Reionization
• CosmoUiT: A Vision Transformer-UNet Hybrid for Fast and Accurate Emulation of 21-cm Maps from the Epoch of Reionization