SKAO HI Intensity Mapping: Blind Foreground Subtraction Challenge
arXiv:2107.10814 · doi:10.1093/mnras/stab3064
Abstract
Neutral Hydrogen Intensity Mapping (HI IM) surveys will be a powerful new probe of cosmology. However, strong astrophysical foregrounds contaminate the signal and their coupling with instrumental systematics further increases the data cleaning complexity. In this work, we simulate a realistic single-dish HI IM survey of a ~deg patch in the MHz range, with both the MID telescope of the SKA Observatory (SKAO) and MeerKAT, its precursor. We include a state-of-the-art HI simulations and explore different foreground models and instrumental effects such as non-homogeneous thermal noise and beam side-lobes. We perform the first Blind Foreground Subtraction Challenge for HI IM on these synthetic data-cubes, aiming to characterise the performance of available foreground cleaning methods with no prior knowledge of the sky components and noise level. Nine foreground cleaning pipelines joined the Challenge, based on statistical source separation algorithms, blind polynomial fitting, and an astrophysical-informed parametric fit to foregrounds. We devise metrics to compare the pipeline performances quantitatively. In general, they can recover the input maps' 2-point statistics within 20 per cent in the range of scales least affected by the telescope beam. However, spurious artefacts appear in the cleaned maps due to interactions between the foreground structure and the beam side-lobes. We conclude that it is fundamental to develop accurate beam deconvolution algorithms and test data post-processing steps carefully before cleaning. This study was performed as part of SKAO preparatory work by the HI IM Focus Group of the SKA Cosmology Science Working Group.
arXiv commentAccepted version for publication in MNRAS
References in corpus (107)
- Scikit-learn: Machine Learning in Python
- SciPy 1.0--Fundamental Algorithms for Scientific Computing in Python
- Array Programming with NumPy
- Astropy: A Community Python Package for Astronomy
- First Year Wilkinson Microwave Anisotropy Probe (WMAP) Observations: Determination of Cosmological Parameters
- The Structure of Cold Dark Matter Halos
- HEALPix -- a Framework for High Resolution Discretization, and Fast Analysis of Data Distributed on the Sphere
- MultiNest: an efficient and robust Bayesian inference tool for cosmology and particle physics
- Multimodal nested sampling: an efficient and robust alternative to MCMC methods for astronomical data analysis
- Cosmology at Low Frequencies: The 21 cm Transition and the High-Redshift Universe
- Astrophysics Source Code Library
- Resolving Cosmic Structure Formation with the Millennium-II Simulation
- MASTER of the CMB Anisotropy Power Spectrum: A Fast Method for Statistical Analysis of Large and Complex CMB Data Sets
- A Full-Sky H-alpha Template for Microwave Foreground Prediction
- A unified pseudo- framework
- Chemical enrichment of the intra-cluster and intergalactic medium in a hierarchical galaxy formation model
- Baryon Acoustic Oscillation Intensity Mapping as a Test of Dark Energy
- Measurement of 21 cm brightness fluctuations at z ~ 0.8 in cross-correlation
- Late-time cosmology with 21cm intensity mapping experiments
- Cosmology with Phase 1 of the Square Kilometre Array; Red Book 2018: Technical specifications and performance forecasts
- Multifrequency analysis of 21 cm fluctuations from the Era of Reionization
- Upper limits on the 21-cm Epoch of Reionization power spectrum from one night with LOFAR
- Determination of z~0.8 neutral hydrogen fluctuations using the 21 cm intensity mapping auto-correlation
- The pre-launch Planck Sky Model: a model of sky emission at submillimetre to centimetre wavelengths
- HI intensity mapping : a single dish approach
- PINOCCHIO: pinpointing orbit-crossing collapsed hierarchical objects in a linear density field
- Canadian Hydrogen Intensity Mapping Experiment (CHIME) Pathfinder
- Separation of anomalous and synchrotron emissions using WMAP polarization data
- An Improved Model of Diffuse Galactic Radio Emission from 10 MHz to 5 THz
- Galaxy assembly, stellar feedback and metal enrichment: the view from the GAEA model
- Foreground Removal using FastICA: A Showcase of LOFAR-EoR
- Lack of clustering in low-redshift 21-cm intensity maps cross-correlated with 2dF galaxy densities
- Coaxing Cosmic 21cm Fluctuations from the Polarized Sky using m-mode Analysis
- HIRAX: A Probe of Dark Energy and Radio Transients
- Blinded challenge for precision cosmology with large-scale structure: results from effective field theory for the redshift-space galaxy power spectrum
- Twenty-one centimeter tomography with foregrounds
- Blind foreground subtraction for intensity mapping experiments
- Neutral hydrogen surveys for high redshift galaxy clusters and proto-clusters
- An accurate tool for the fast generation of dark matter halo catalogs
- The Scale of the Problem : Recovering Images of Reionization with GMCA
- Redshift Space Distortion of the 21cm Background from the Epoch of Reionization I: Methodology Re-examined
- A Determination of the Spectral Index of Galactic Synchrotron Emission in the 1-10 GHz range
- The 1.28 GHz MeerKAT DEEP2 Image
- Statistical 21-cm Signal Separation via Gaussian Process Regression Analysis
- Highest Redshift Image of Neutral Hydrogen in Emission: A CHILES Detection of a Starbursting Galaxy at z=0.376
- A ground-based 21cm Baryon acoustic oscillation survey
- Using HI to probe large scale structures at z ~ 3
- Anticipated Performance of the Square Kilometre Array -- Phase 1 (SKA1)
- HI as a Probe of the Large Scale Structure in the Post-Reionization Universe
- The effect of foreground subtraction on cosmological measurements from Intensity Mapping
- Planck 2015 results. XXV. Diffuse low-frequency Galactic foregrounds
- Elemental Abundances in Milky Way-like Galaxies from a Hierarchical Galaxy Formation Model
- H-based star formation laws in hierarchical models of galaxy formation
- Full Sky Study of Diffuse Galactic Emission at Decimeter Wavelengths
- Interpreting the unresolved intensity of cosmologically redshifted line radiation
- The contribution of the Unresolved Extragalactic Radio Sources to the Brightness Temperature of the sky
- HI intensity mapping with MeerKAT: Calibration pipeline for multi-dish autocorrelation observations
- Fast simulations for intensity mapping experiments
- Improved foreground removal in GMRT 610 MHz observations towards redshifted 21-cm tomography
- Simulations for single-dish intensity mapping experiments
- Improving fast generation of halo catalogs with higher-order Lagrangian perturbation theory
- 21cm foregrounds and polarization leakage: a user's guide on cleaning and mitigation strategies
- Erasing the Milky Way: new cleaning technique applied to GBT intensity mapping data
- Baryonic acoustic oscillations from 21cm intensity mapping: the Square Kilometre Array case
- Extracting HI cosmological signal with Generalized Needlet Internal Linear Combination
- Joint Planck and WMAP CMB Map Reconstruction
- MeerKLASS: MeerKAT Large Area Synoptic Survey
- Recovery of 21 cm intensity maps with sparse component separation
- The atomic Hydrogen content of the post-reionization Universe
- deep21: a Deep Learning Method for 21cm Foreground Removal
- The 21cm Power Spectrum After Reionization
- Primary beam effects of radio astronomy antennas -- II. Modelling the MeerKAT L-band beam
- Dark Energy Survey Year 3 Results: High-precision measurement and modeling of galaxy-galaxy lensing
- Comparing Foreground Removal Techniques for Recovery of the LOFAR-EoR 21cm Power Spectrum
- Warm dark matter signatures on the 21cm power spectrum: Intensity mapping forecasts for SKA
- PINOCCHIO and the hierarchical build-up of dark matter haloes
- Image Analysis for Cosmology: Results from the GREAT10 Star Challenge
- Impact of Foregrounds on HI Intensity Mapping Cross-Correlations with Optical Surveys
- Hi intensity mapping with MeerKAT: Primary beam effects on foreground cleaning
- Impact of Simulated 1/f Noise for HI Intensity Mapping Experiments
- GMRT observation towards detecting the Post-reionization 21-cm signal
- 21 cm observation of LSS at z~1 Instrument sensitivity and foreground subtraction
- HI-selected Galaxies in Hierarchical Models of Galaxy Formation and Evolution
- Sparse component separation for accurate CMB map estimation
- Cosmological parameter forecasts for HI intensity mapping experiments using the angular power spectrum
- Forecast for FAST: from Galaxies Survey to Intensity Mapping
- HI intensity mapping with MeerKAT: 1/f noise analysis
- Gaussian Process Regression for foreground removal in HI intensity mapping experiments
- Intensity mapping with neutral hydrogen and the Hidden Valley simulations
- Detecting the relativistic bispectrum in 21cm intensity maps
- Update on the BINGO 21cm intensity mapping experiment
- Potential Impact of Global Navigation Satellite Services on Total Power HI Intensity Mapping Surveys
- A novel method for component separation of extended sources in X-ray astronomy
- Intensity Mapping in the Presence of Foregrounds and Correlated Continuum Emission
- Simulations of Galactic polarized synchrotron emission for Epoch of Reionization observations
- Exploring the Cosmic Reionization Epoch in Frequency Space: An Improved Approach to Remove the Foreground in 21 cm Tomography
- First multi-redshift limits on post-Epoch of Reionization (post-EoR) 21 cm signal from z = 1.96 - 3.58 using uGMRT
- The HI intensity mapping bispectrum including observational effects
- How to Identify and Separate Bright Galaxy Clusters from the Low-frequency Radio Sky?
- Cleaning foregrounds from single-dish 21cm intensity maps with Kernel Principal Component Analysis
- HIR4: cosmology from a simulated neutral hydrogen full sky using Horizon Run 4
- Imprints of non-standard Dark Energy and Dark Matter Models on the 21cm Intensity Map Power Spectrum
- Measuring ultra-large scale effects in the presence of 21cm intensity mapping foregrounds
- Recovering 21-cm signal from simulated FAST intensity maps
- Impact of noise on cosmological parameter constraints for SKA intensity mapping
- Sparsity Based Recovery of Galactic Binaries Gravitational Waves
- Joint deconvolution and unsupervised source separation for data on the sphere
Cited by in corpus (46)
- HI intensity mapping with MeerKAT: power spectrum detection in cross-correlation with WiggleZ galaxies
- HI constraints from the cross-correlation of eBOSS galaxies and Green Bank Telescope intensity maps
- Cosmology Intertwined: A Review of the Particle Physics, Astrophysics, and Cosmology Associated with the Cosmological Tensions and Anomalies
- The foreground transfer function for HI intensity mapping signal reconstruction: MeerKLASS and precision cosmology applications
- Gravitational waves HI intensity mapping: cosmological and astrophysical applications
- Eliminating Primary Beam Effect in Foreground Subtraction of Neutral Hydrogen Intensity Mapping Survey with Deep Learning
- The Second Radio Synchrotron Background Workshop: Conference Summary and Report
- Modeling HI at the field level
- Detecting the power spectrum turnover with HI intensity mapping
- Cosmological constraints from the power spectrum and bispectrum of 21cm intensity maps
- Constraining primordial non-Gaussianity by combining next-generation galaxy and 21 cm intensity mapping surveys
- Cosmological Probes of Structure Growth and Tests of Gravity
- Detecting the HI Power Spectrum in the Post-Reionization Universe with SKA-Low
- Eliminating polarization leakage effect for neutral hydrogen intensity mapping with deep learning
- Multipole expansion for 21cm Intensity Mapping power spectrum: forecasted cosmological parameters estimation for the SKA Observatory
- Interferometric HI intensity mapping: perturbation theory predictions and foreground removal effects
- Emission-line Stacking of 21cm Intensity Maps with MeerKLASS: Inference Pipeline and Application to the L-band Deep-field Data
- Radio-optical synergies at high redshift to constrain primordial non-Gaussianity
- Validating galaxy clustering models with Fixed & Paired and Matched-ICs simulations: application to Primordial Non-Gaussianities
- Cross-Correlation Forecast of CSST Spectroscopic Galaxy and MeerKAT Neutral Hydrogen Intensity Mapping Surveys
- Probing Ultra-light Axion Dark Matter from 21cm Tomography using Convolutional Neural Networks
- Unveiling the Universe with Emerging Cosmological Probes
- Sensitivity of Bayesian 21 cm power spectrum estimation to foreground model errors
- Baryon acoustic oscillations from HI intensity mapping: the importance of cross-correlations in the monopole and quadrupole
- Fast Simulation of Cosmological Neutral Hydrogen based on the Halo Model
- HI Intensity Mapping with the MIGHTEE Survey: First Results of the HI Power Spectrum
- A wide-angle formulation of foreground filters for HI intensity mapping
- Prospects for cosmological research with the FAST array: 21-cm intensity mapping survey observation strategies
- Constraining primordial non-Gaussianity by combining photometric galaxy and 21cm intensity mapping surveys
- Multi-tracing the primordial Universe with future surveys
- Exploring One-point Statistics in HERA Phase I Data: Effects of Foregrounds and Systematics on Measuring One-Point Statistics
- Application of Physics-Informed Neural Networks in Removing Telescope Beam Effects
- FAST Drift Scan Survey for HI Intensity Mapping. II. Stacking-based Beam Construction of the 19-feed Array at GHz
- Primary Beam Chromaticity in HIRAX: I. Characterization from Simulations and Power Spectrum Implications
- Foreground removal and angular power spectrum estimation of 21 cm signal using harmonic space ILC method
- Extracting the Epoch of Reionization Signal with 3D U-Net Neural Networks Using Data-driven Systematic Effect Model
- Needlets and foreground removal for SKAO hydrogen intensity maps
- Foreground removal in HI 21 cm intensity mapping under frequency-dependent beam distortions
- Exploring HI Galaxy Redshift Survey Strategies for the FAST Core Array Interferometry
- Impact of astrophysical effects on the dark matter mass constraint with 21cm intensity mapping
- Constraining the growth rate on linear scales by combining SKAO and DESI surveys
- Probing the large-scale structure with 21cm-galaxy cross-bispectrum: Estimates from simulations and forecasts for upcoming cosmological surveys
- Simulating a full-sky high resolution Galactic synchrotron spectral index map using neural networks
- Seeing Wiggles without Seeing Wiggles: BAO Recovery in 21 cm Intensity Mapping with Deep Learning
- Clustering redshifts with the 21cm-galaxy cross-bispectrum
- Measurements of the diffuse Galactic synchrotron spectral index and curvature from MeerKLASS pilot data