Data Analysis for Precision 21 cm Cosmology

Abstract

The redshifted 21 cm line is an emerging tool in cosmology, in principle permitting three-dimensional surveys of our Universe that reach unprecedentedly large volumes, previously inaccessible length scales, and hitherto unexplored epochs of our cosmic timeline. Large radio telescopes have been constructed for this purpose, and in recent years there has been considerable progress in transforming 21 cm cosmology from a field of considerable theoretical promise to one of observational reality. Increasingly, practitioners in the field are coming to the realization that the success of observational 21cm cosmology will hinge on software algorithms and analysis pipelines just as much as it does on careful hardware design and telescope construction. This review provides a pedagogical introduction to state-of-the-art ideas in 21 cm data analysis, covering a wide variety of steps in a typical analysis pipeline, from calibration to foreground subtraction to mapmaking to power spectrum estimation to parameter estimation.

105 pages, 16 figures; Invited review for PASP; v2: replaced with accepted PASP version

References in corpus (116)

• Cosmology at Low Frequencies: The 21 cm Transition and the High-Redshift Universe
• The Hubble Hunter's Guide
• GaLactic and Extragalactic All-sky Murchison Widefield Array (GLEAM) survey I: A low-frequency extragalactic catalogue
• Multi-Scale CLEAN deconvolution of radio synthesis images
• Baryon Acoustic Oscillation Intensity Mapping as a Test of Dark Energy
• The morphology of HII regions during reionization
• Measurement of 21 cm brightness fluctuations at z ~ 0.8 in cross-correlation
• Late-time cosmology with 21cm intensity mapping experiments
• Post-correlation radio frequency interference classification methods
• A Second Source of Repeating Fast Radio Bursts
• How accurately can 21 cm tomography constrain cosmology?
• The Epoch of Reionization Window: I. Mathematical Formalism
• Canadian Hydrogen Intensity Mapping Experiment (CHIME) Pathfinder
• The Epoch of Reionization Window: II. Statistical Methods for Foreground Wedge Reduction
• Simulating Cosmic Reionization at Large Scales II: the 21-cm Emission Features and Statistical Signals
• Signature of Excess Radio Background in the 21-cm Global Signal and Power Spectrum
• Observations of Fast Radio Bursts at Frequencies down to 400 Megahertz
• Real-Time Calibration of the Murchison Widefield Array
• A Method for 21cm Power Spectrum Estimation in the Presence of Foregrounds
• Long troughs in the Lyman-$\alpha$ forest below redshift 6 due to islands of neutral hydrogen
• HIRAX: A Probe of Dark Energy and Radio Transients
• The Neutral Hydrogen Cosmological Mass Density at z=5
• First Season MWA EoR Power Spectrum Results at Redshift 7
• Detecting the Rise and Fall of 21 cm Fluctuations with the Murchison Widefield Array
• The 21cm angular-power spectrum from the dark ages
• Line-Intensity Mapping: 2017 Status Report
• LOFAR/H-ATLAS: A deep low-frequency survey of the Herschel-ATLAS North Galactic Pole field
• The Fast Fourier Transform Telescope
• The Global 21-cm Signal in the Context of the High-z Galaxy Luminosity Function
• Testing for calibration systematics in the EDGES low-band data using Bayesian model selection
• Improving the Epoch of Reionization Power Spectrum Results from Murchison Widefield Array Season 1 Observations
• Three-Year Wilkinson Microwave Anisotropy Probe (WMAP) Observations: Foreground Polarization
• The Scale of the Problem : Recovering Images of Reionization with GMCA
• An Improved Method for 21cm Foreground Removal
• Large-scale BAO signatures of the smallest galaxies
• Cosmic Calibration: Constraints from the Matter Power Spectrum and the Cosmic Microwave Background
• Spectral Index of the Diffuse Radio Background Measured From 100 to 200 MHz
• The 21 cm Power Spectrum from the Cosmic Dawn: First Results from the OVRO-LWA
• Toward Empirical Constraints on the Global Redshifted 21 cm Brightness Temperature During the Epoch of Reionization
• The multi-frequency angular power spectrum of the epoch of reionization 21 cm signal
• Fast Holographic Deconvolution: a new technique for precision radio interferometry
• Non-parametric foreground subtraction for 21cm epoch of reionization experiments
• Implications of a transition in the dark energy equation of state for the $H_0$ and ${\sigma }_8$ tensions
• MITEoR: A Scalable Interferometer for Precision 21 cm Cosmology
• Cosmic Calibration
• Foreground Model and Antenna Calibration Errors in the Measurement of the Sky-Averaged λ21 cm Signal at z~20
• Distributed Radio Interferometric Calibration
• Forecasts on the Dark Energy and Primordial Non-Gaussianity Observations with the Tianlai Cylinder Array
• Emulation of reionization simulations for Bayesian inference of astrophysics parameters using neural networks
• Intensity mapping of [CII] from early galaxies
• Will point sources spoil 21 cm tomography?
• Multi-redshift limits on the 21cm power spectrum from PAPER
• Characterisation of the ionosphere above the Murchison Radio Observatory using the Murchison Widefield Array
• Mitigating Internal Instrument Coupling II: A Method Demonstration with the Hydrogen Epoch of Reionization Array
• Mitigating Internal Instrument Coupling for 21 cm Cosmology I: Temporal and Spectral Modeling in Simulations
• Improved Measurement of the Spectral Index of the Diffuse Radio Background Between 90 and 190 MHz
• Primordial gravity wave fossils and their use in testing inflation
• Higher Order Contributions to the 21 cm Power Spectrum
• Robust Velocity-induced Acoustic Oscillations at Cosmic Dawn
• Results from EDGES High-Band: III. New Constraints on Parameters of the Early Universe
• A Standard Ruler at Cosmic Dawn
• The Impact of Foregrounds on Redshift Space Distortion Measurements With the Highly-Redshifted 21 cm Line
• The effect of non-Gaussianity on error predictions for the Epoch of Reionization (EoR) 21-cm power spectrum
• Detection and extraction of signals from the epoch of reionization using higher order one-point statistics
• Spectral Index of the Diffuse Radio Background Between 50 and 100 MHz
• Visibility based angular power spectrum estimation in low frequency radio interferometric observations
• A Space-Based Observational Strategy for Characterizing the First Stars and Galaxies Using the Redshifted 21-cm Global Spectrum
• SARAS measurement of the Radio Background at long wavelengths
• Optimizing Sparse RFI Prediction using Deep Learning
• The Radio Scream from Black Holes at Cosmic Dawn: A Semi-Analytic Model for the Impact of Radio Loud Black-Holes on the 21 cm Global Signal
• Spherical Harmonic Analyses of Intensity Mapping Power Spectra
• Radio background and IGM heating due to Pop III supernovae explosions
• Sensitivity for 21cm Bispectrum from Epoch of Reionization
• Mapmaking for Precision 21 cm Cosmology
• The correlation between the distribution of galaxies and 21cm emission at high redshifts
• Adding Context to JWST Surveys with Current and Future 21cm Radio Observations
• Studying 21cm power spectrum with one-point statistics
• Astrophysics and Cosmology with Line-Intensity Mapping
• The Difference PDF of 21-cm Fluctuations: A Powerful Statistical Tool for Probing Cosmic Reionization
• Erasing the variable: Empirical foreground discovery for global 21 cm spectrum experiments
• GMOSS: All-sky model of spectral radio brightness based on physical components and associated radiative processes
• PUMA: The Positional Update and Matching Algorithm
• Science with the Murchison Widefield Array: Phase I Results and Phase II Opportunities
• The Global Magneto-Ionic Medium Survey: Polarimetry of the Southern Sky from 300 to 480 MHz
• A High Reliability Survey of Discrete Epoch of Reionization Foreground Sources in the MWA EoR0 Field
• Gridded and direct Epoch of Reionisation bispectrum estimates using the Murchison Widefield Array
• Update on the BINGO 21cm intensity mapping experiment
• Measuring the growth of structure with intensity mapping surveys
• Contamination of the Epoch of Reionization power spectrum in the presence of foregrounds
• The impact of non-Gaussianity on the error covariance for observations of the Epoch of Reionization 21-cm power spectrum
• Joint estimation of the Epoch of Reionization power spectrum and foregrounds
• RFI detection by automated feature extraction and statistical analysis
• Intensity Mapping in the Presence of Foregrounds and Correlated Continuum Emission
• The Ooty Wide Field Array
• A Tapered Gridded Estimator (TGE) for the Multi-Frequency Angular Power Spectrum (MAPS) and the Cosmological HI 21-cm Power Spectrum
• How to Search for Islands of Neutral Hydrogen in the $z\sim 5.5$ IGM
• Lunar occultation of the diffuse radio sky: LOFAR measurements between 35 and 80 MHz
• Canceling out intensity mapping foregrounds
• Studying the morphology of reionisation with the triangle correlation function of phases
• Selection between foreground models for global 21-cm experiments
• A First Search for Prompt Radio Emission from a Gravitational-Wave Event
• An Improved Statistical Point-Source Foreground Model for the Epoch of Reionization
• Anatomy of Cosmic Tidal Reconstruction
• Delay Spectrum with Phase-Tracking Arrays: Extracting the HI power spectrum from the Epoch of Reionization
• Fundamental uncertainty levels of 21cm power spectra from a delay analysis
• Bayesian Model Selection with Future 21cm Observations of The Epoch of Reionisation
• CDIM: Cosmic Dawn Intensity Mapper Final Report
• A Real-Time, All-Sky, High Time Resolution, Direct Imager for the Long Wavelength Array
• Comparison of observing modes for statistical estimation of the 21cm signal from the Epoch of Reionisation
• Calibration of Quasi-Redundant Interferometers
• The Breakdown Scale of HI Bias Linearity
• 21-cm signatures of residual HI inside cosmic HII regions during reionization
• Extracting the Global Signal from 21-cm Fluctuations: the Multi-Tracer Approach
• Robust statistics toward detection of the 21 cm signal from the Epoch of Reionisation
• Measuring the History of Cosmic Reionization using the 21-cm Difference PDF
• Pico: Parameters for the Impatient Cosmologist

Cited by in corpus (147)

• Improved upper limits on the 21-cm signal power spectrum of neutral hydrogen at $\mathbf{z}\approx 9.1$ from LOFAR
• First Results from HERA Phase I: Upper Limits on the Epoch of Reionization 21 cm Power Spectrum
• Improved Constraints on the 21 cm EoR Power Spectrum and the X-Ray Heating of the IGM with HERA Phase I Observations
• HERA Phase I Limits on the Cosmic 21-cm Signal: Constraints on Astrophysics and Cosmology During the Epoch of Reionization
• Constraining primordial black holes as dark matter using the global 21-cm signal with X-ray heating and excess radio background
• Probing the Small-Scale Matter Power Spectrum with Large-Scale 21-cm Data
• 21cm foregrounds and polarization leakage: a user's guide on cleaning and mitigation strategies
• deep21: a Deep Learning Method for 21cm Foreground Removal
• Redundant-Baseline Calibration of the Hydrogen Epoch of Reionization Array
• Fuzzy Dark Matter and the 21cm Power Spectrum
• Gaussian Process Foreground Subtraction and Power Spectrum Estimation for 21 cm Cosmology
• The Tianlai Dish Pathfinder Array: design, operation and performance of a prototype transit radio interferometer
• GLOBALEMU: A novel and robust approach for emulating the sky-averaged 21-cm signal from the cosmic dawn and epoch of reionisation
• An Effective Model for the Cosmic-Dawn 21-cm Signal
• Recovering the Wedge Modes Lost to 21-cm Foregrounds
• DAYENU: A Simple Filter of Smooth Foregrounds for Intensity Mapping Power Spectra
• Closing the window on fuzzy dark matter with the 21cm signal
• The Tianlai Cylinder Pathfinder Array: System Functions and Basic Performance Analysis
• Gaussian Process Regression for foreground removal in HI intensity mapping experiments
• Exploring the likelihood of the 21-cm power spectrum with simulation-based inference
• 21cmEMU: an emulator of 21cmFAST summary observables
• Hydrogen Epoch of Reionization Array (HERA) Phase II Deployment and Commissioning
• 21cmVAE: A Very Accurate Emulator of the 21-cm Global Signal
• Array Element Coupling in Radio Interferometry I: A Semi-Analytic Approach
• Quantifying Excess Power from Radio Frequency Interference in Epoch of Reionization Measurements
• New EoR Power Spectrum Limits From MWA Phase II Using the Delay Spectrum Method and Novel Systematic Rejection
• Future Constraints on the Reionization History and the Ionizing Sources from Gamma-ray Burst Afterglows
• Validation of the HERA Phase I Epoch of Reionization 21 cm Power Spectrum Software Pipeline
• A direct detection of neutral hydrogen intensity mapping on Mpc scales at $z\approx 0.32$ and $z\approx 0.44$
• First Constraints on the Epoch of Reionization Using the non-Gaussianity of the Kinematic Sunyaev-Zel{'}dovich Effect from the South Pole Telescope and {\it Herschel}-SPIRE Observations
• Assessing the impact of two independent direction-dependent calibration algorithms on the LOFAR 21-cm signal power spectrum
• Cosmological constraints from the power spectrum and bispectrum of 21cm intensity maps
• LIMFAST. II. Line Intensity Mapping as a Probe of High-Redshift Galaxy Formation
• LIMFAST. I. A Semi-Numerical Tool for Line Intensity Mapping
• Deep learning approach for identification of HII regions during reionization in 21-cm observations -- II. foreground contamination
• 21cmfish: Fisher-matrix framework for fast parameter forecasts from the cosmic 21-cm signal
• The Impact of Beam Variations on Power Spectrum Estimation for 21-cm Cosmology I: Simulations of Foreground Contamination for HERA
• An Effective Bias Expansion for 21 cm Cosmology in Redshift Space
• Prospects for 21cm-Galaxy Cross-Correlations with HERA and the Roman High-Latitude Survey
• Evidence of Ultra-faint Radio Frequency Interference in Deep 21~cm Epoch of Reionization Power Spectra with the Murchison Widefield Array
• Systematic uncertainties in models of the cosmic dawn
• Observing the Reionization : Effect of Calibration and Position Errors on Realistic Observation Conditions
• Constraining the Epoch of Reionization With Highly Dispersed Fast Radio Bursts
• Methods of Error Estimation for Delay Power Spectra in $21\text{cm}$ Cosmology
• 21cmFirstCLASS I. Cosmological tool for $\Lambda$CDM and beyond
• Exploring the cosmic dawn and epoch of reionization with 21cm line
• A General Bayesian Framework to Account for Foreground Map Errors in Global 21-cm Experiments
• Quantifying EoR delay spectrum contamination from diffuse radio emission
• 21cmSense v2: A modular, open-source 21cm sensitivity calculator
• The Tianlai dish array low-z surveys forecasts
• Machine-learning recovery of foreground wedge-removed 21-cm light cones for high-$z$ galaxy mapping
• A Bayesian approach to high fidelity interferometric calibration II: demonstration with simulated data
• A Semi-blind PCA-based Foreground Subtraction Method for 21 cm Intensity Mapping
• Quantifying Density-Ionization Correlations with the 21cm Power Spectrum
• Impact of instrument and data characteristics in the interferometric reconstruction of the 21 cm power spectrum
• Forecasts and Statistical Insights for Line Intensity Mapping Cross-Correlations: A Case Study with 21cm x [CII]
• Persistent topology of the reionization bubble network. II: Evolution & Classification
• Extracting the Signal of Cosmic String Wakes from 21-cm Observations
• Exploring the consequences of chromatic data excision in 21-cm Epoch of Reionization power spectrum observations
• The First Billion Years in Seconds: An Effective Model for the 21-cm Signal with Population III Stars
• On the general nature of 21cm-Lyman-$\alpha$ emitters cross-correlations during reionisation
• A multi-tracer empirically-driven approach to line-intensity mapping lightcones
• HI intensity mapping with the MIGHTEE survey: power spectrum estimates
• All Sky Modelling Requirements for Bayesian 21 cm Power Spectrum Estimation with BayesEoR
• Reflections and Standing Waves on the Tianlai Cylinder Array
• A galaxy-free phenomenological model for the 21-cm power spectrum during reionization
• SKA-Low Intensity Mapping Pathfinder Updates: Deeper 21 cm Power Spectrum Limits from Improved Analysis Frameworks
• Bayesian evidence-driven diagnosis of instrumental systematics for sky-averaged 21-cm cosmology experiments
• Joint constraints on reionization: a framework for combining the global 21cm signal and the kinetic Sunyaev-Zel'dovich effect
• HI intensity mapping with MeerKAT: forecast for delay power spectrum measurement using interferometer mode
• Characterization Of Inpaint Residuals In Interferometric Measurements of the Epoch Of Reionization
• Synergizing 21 cm and sub-millimetre surveys during reionization: new empirical insights
• Interferometric HI intensity mapping: perturbation theory predictions and foreground removal effects
• 1/f Noise Analysis for FAST HI Intensity Mapping Drift-Scan Experiment
• Subtracting the kinetic Sunyaev-Zeldovich effect from the cosmic microwave background with surveys of large-scale structure
• Direct Optimal Mapping for 21cm Cosmology: A Demonstration with the Hydrogen Epoch of Reionization Array
• Spectral modelling of Cygnus A between 110 and 250 MHz. Impact on the LOFAR 21-cm signal power spectrum
• 21-cm foreground removal using AI and frequency-difference technique
• Ionospheric contributions to the excess power in high-redshift 21-cm power-spectrum observations with LOFAR
• Antisymmetric Cross-correlation between H I and CO Line Intensity Maps as a New Probe of Cosmic Reionization
• Impact of extragalactic point sources on the low-frequency sky spectrum and cosmic dawn global 21-cm measurements
• Using the sample variance of 21cm maps as a tracer of the ionisation topology
• Detecting galaxy-21-cm cross-correlation during reionization
• Delay-Weighted Calibration: Precision Calibration for 21 cm Cosmology with Resilience to Sky Model Error
• First upper limits on the 21-cm signal power spectrum of neutral hydrogen at $z=9.16$ from the LOFAR 3C196 field
• The mass distribution of the first stars can be determined via the 21-cm signal
• Modeling Post-Reionization HI Distributions in Fuzzy Dark Matter Cosmologies Using Conditional Normalizing Flows
• Why and When to Expect Gaussian Error Distributions in Epoch of Reionization 21-cm Power Spectrum Measurements
• Unveiling the Universe with Emerging Cosmological Probes
• Cosmology at high redshift -- a probe of fundamental physics
• Lyman Alpha Emitters and the 21cm Power Spectrum as Probes of Density-Ionization Correlation in the Epoch of Reionization
• Towards Optimal Foreground Mitigation Strategies for Interferometric HI Intensity Mapping in the Low-Redshift Universe
• Measurements of one-point statistics in 21 cm intensity maps via foreground avoidance strategy
• Towards Accurate Modeling of Line-Intensity Mapping One-Point Statistics: Including Extended Profiles
• Rapid and Late Cosmic Reionization Driven by Massive Galaxies: a Joint Analysis of Constraints from 21-cm, Lyman Line & CMB Data Sets
• Altitude Estimation of Radio Frequency Interference Sources via Interferometric Near Field Corrections
• A Differentiable, End-to-End Forward Model for 21 cm Cosmology: Estimating the Foreground, Instrument, and Signal Joint Posterior
• Sensitivity of Bayesian 21 cm power spectrum estimation to foreground model errors
• A Multi-messenger view of Cosmic Dawn: Conquering the Final Frontier
• A statistical framework for recovering intensity mapping autocorrelations from crosscorrelations
• Understanding spectral artefacts in SKA-LOW 21-cm cosmology experiments: the impact of cable reflections
• Statistical estimation of full-sky radio maps from 21cm array visibility data using Gaussian constrained realizations
• Bayesian evidence for flux scale errors in Galactic synchrotron maps
• Automated Detection of Antenna Malfunctions in Large-N Interferometers: A Case Study with the Hydrogen Epoch of Reionization Array
• Fast Simulation of Cosmological Neutral Hydrogen based on the Halo Model
• Linear Polarization of the 21 cm Line from the Epoch of Reionization
• Ionospheric effect on the synthetic Epoch of Reionization observations with the SKA1-Low
• Constraining a Model of the Radio Sky Below 6 MHz Using the Parker Solar Probe/FIELDS Instrument in Preparation for Upcoming Lunar-based Experiments
• A general Bayesian model-validation framework based on null-test evidence ratios, with an example application to global 21-cm cosmology
• Reconstructing the Long-wavelength Matter Density Fluctuation Modes from the Scalar-Type Clustering Fossils
• Low-Frequency Radio Recombination Lines Away From the Inner Galactic Plane
• 21 cm Intensity Mapping with the DSA-2000
• A Minimal Space Interferometer Configuration for Imaging at Low Radio Frequencies
• Prospects for probing dark matter particles and primordial black holes with the Hongmeng mission using the 21 cm global spectrum at cosmic dawn
• A novel radio imaging method for physical spectral index modelling
• The Statistics of Negative Power Spectrum Systematics in some 21 cm Analyses
• Low-cost, Low-loss, Ultra-wideband Compact Feed for Interferometric Radio Telescopes
• Bayesian jackknife tests with a small number of subsets: Application to HERA 21cm power spectrum upper limits
• Calibration Schemes with $\mathcal{O}(N\log N)$ Scaling for Large-N Radio Interferometers Built on a Regular Grid
• The Hydrogen Mixing Portal, Its Origins, and Its Cosmological Effects
• Reionization Parameter Inference from 3D Minkowski Functionals of the 21 cm Signals
• The Hydrogen Intensity and Real-time Analysis eXperiment: 256-Element Array Status and Overview
• The 21cm-galaxy cross-correlation: Realistic forecast for 21cm signal detection and reionisation constraints
• Astrophysical uncertainties challenge 21-cm forecasts: A primordial black hole case study
• Drone Beam Mapping of the TONE Radio Dish Array
• AlgoSCR: An algorithm for Solar Contamination Removal from radio interferometric data
• Direct correlation of line intensity mapping and CMB lensing from evolution along the lightcone
• Removal of interloper contamination to line-intensity maps using correlations with ancillary tracers of the large-scale structure
• An Alcock-Paczynski Test on Reionization Bubbles for Cosmology
• Foreground removal and angular power spectrum estimation of 21 cm signal using harmonic space ILC method
• A semi-cosmographic approach to study cosmological evolution in phase space
• The impact of lunar topography on the 21-cm power spectrum for grid-based arrays : Insights for the Dark-ages EXplorer (DEX)
• Deep learning with hybrid frequency differencing and principal component analysis for 21-cm foreground and beam mitigation
• Probing the warm dark matter mass with [C II] intensity mapping
• Beyond the Power Spectrum: A New Framework for Non-Stationary Fields with Applications to Light-Cone Effects in Line Intensity Mapping
• Seeing Wiggles without Seeing Wiggles: BAO Recovery in 21 cm Intensity Mapping with Deep Learning
• Foreground Subtraction with a Tensor-Based Oriented Singular Value Decomposition Method for HI Experiments
• Robust Intensity Mapping Analysis against Foregrounds for the Epoch of Reionization
• Establishing a relationship between the cosmological 21 cm power spectrum and interferometric closure phases
• Testing the growth of cosmic structures during the Dark Ages
• Calibrating Bayesian Tension Statistics using Neural Ratio Estimation
• Simulation based inference of the ionization history from the 2D 21 cm power spectrum
• A Bayesian approach to high fidelity interferometric calibration I: mathematical formalism
• Exploring delaying and heating effects on the 21-cm signature of fuzzy dark matter
• Implications of the cosmological 21-cm absorption profile for high-redshift star formation and deep JWST surveys
• Power spectrum multipoles and clustering wedges during the Epoch of Reionization
• Exploring the Model Dependence of MCMC-Based 21 cm Power Spectrum Parameter Constraints