The impact of interference excision on 21-cm Epoch of Reionization power spectrum analyses

Abstract

We investigate the implications of interference detection for experiments that are pursuing a detection of the redshifted 21-cm signals from the Epoch of Reionization. Interference detection causes samples to be sporadically flagged and rejected. As a necessity to reduce the data volume, flagged samples are typically (implicitly) interpolated during time or frequency averaging or uv-gridding. This so-far unexplored systematic biases the 21-cm power spectrum, and it is important to understand this bias for current 21-cm experiments as well as the upcoming SKA Epoch of Reionization experiment. We analyse simulated data using power spectrum analysis and Gaussian process regression. We find that the combination of flagging and averaging causes tiny spectral fluctuations, resulting in `flagging excess power'. This excess power does not substantially average down over time and, without extra mitigation techniques, can exceed the power of realistic models of the 21-cm reionization signals in LOFAR observations. We mitigate the bias by i) implementing a novel way to average data using a Gaussian-weighted interpolation scheme; ii) using unitary instead of inverse-variance weighting of visibilities; and iii) using low-resolution forward modelling of the data. After these modifications, which have been integrated in the LOFAR EoR processing pipeline, the excess power reduces by approximately three orders of magnitude, and is no longer preventing a detection of the 21-cm signals.

arXiv comment11 pages, 8 figures, accepted for publication in MNRAS

References in corpus (46)

• LOFAR: The LOw-Frequency ARray
• Cosmology at Low Frequencies: The 21 cm Transition and the High-Redshift Universe
• WSClean: an implementation of a fast, generic wide-field imager for radio astronomy
• GaLactic and Extragalactic All-sky Murchison Widefield Array (GLEAM) survey I: A low-frequency extragalactic catalogue
• Cosmological Parameter Estimation Using 21 cm Radiation from the Epoch of Reionization
• A morphological algorithm for improving radio-frequency interference detection
• Bright Source Subtraction Requirements For Redshifted 21 cm Measurements
• Four Fundamental Foreground Power Spectrum Shapes for 21 cm Cosmology Observations
• Post-correlation radio frequency interference classification methods
• A broadband flux scale for low frequency radio telescopes
• Upper limits on the 21-cm Epoch of Reionization power spectrum from one night with LOFAR
• Foreground simulations for the LOFAR - Epoch of Reionization Experiment
• 21CMMC: an MCMC analysis tool enabling astrophysical parameter studies of the cosmic 21 cm signal
• Imaging the Epoch of Reionization: limitations from foreground confusion and imaging algorithms
• A Sensitivity and Array-Configuration Study for Measuring the Power Spectrum of 21cm Emission from Reionization
• Calibration Requirements for Detecting the 21 cm Epoch of Reionization Power Spectrum and Implications for the SKA
• Extragalactic Peaked-Spectrum Radio Sources at Low Frequencies
• First Season MWA EoR Power Spectrum Results at Redshift 7
• The low-frequency environment of the Murchison Widefield Array: radio-frequency interference analysis and mitigation
• On the Direct Detectability of the Cosmic Dark Ages: 21-cm Emission from Minihalos
• Initial deep LOFAR observations of Epoch of Reionization windows: I. The North Celestial Pole
• Power Spectrum Sensitivity and the Design of Epoch of Reionization Observatories
• The Scale of the Problem : Recovering Images of Reionization with GMCA
• Radio Interferometric Calibration Using The SAGE Algorithm
• CHIPS: The Cosmological HI Power Spectrum Estimator
• Statistical 21-cm Signal Separation via Gaussian Process Regression Analysis
• Foregrounds for observations of the cosmological 21 cm line: II. Westerbork observations of the fields around 3C196 and the North Celestial Pole
• The Impact of Modeling Errors on Interferometer Calibration for 21 cm Power Spectra
• Systematic biases in low frequency radio interferometric data due to calibration: the LOFAR EoR case
• Non-parametric foreground subtraction for 21cm epoch of reionization experiments
• Calibration of Low-Frequency, Wide-Field Radio Interferometers Using Delay/Delay-Rate Filtering
• The LOFAR radio environment
• Will point sources spoil 21 cm tomography?
• Parametrising Epoch of Reionization foregrounds: A deep survey of low-frequency point-source spectra with the MWA
• The Murchison Widefield Array 21 cm Power Spectrum Analysis Methodology
• The 154 MHz radio sky observed by the Murchison Widefield Array: noise, confusion and first source count analyses
• FLAGCAL:A flagging and calibration package for radio interferometric data
• Compression of interferometric radio-astronomical data
• Software Holography: Interferometric Data Analysis for the Challenges of Next Generation Observatories
• SERPent: Automated reduction and RFI-mitigation software for e-MERLIN
• A High Reliability Survey of Discrete Epoch of Reionization Foreground Sources in the MWA EoR0 Field
• A high resolution foreground model for the MWA EoR1 field: model and implications for EoR power spectrum analysis
• Automated Editing of Radio Interferometer Data with Pieflag
• RFI detection by automated feature extraction and statistical analysis
• Deconvolving the Wedge: Maximum-Likelihood Power Spectra via Spherical-Wave Visibility Modeling
• Building models for extended radio sources: implications for Epoch of Reionisation science

Cited by in corpus (45)

• Improved upper limits on the 21-cm signal power spectrum of neutral hydrogen at $\mathbf{z}\approx 9.1$ from LOFAR
• First Results from HERA Phase I: Upper Limits on the Epoch of Reionization 21 cm Power Spectrum
• Improving the Epoch of Reionization Power Spectrum Results from Murchison Widefield Array Season 1 Observations
• Mitigating Internal Instrument Coupling II: A Method Demonstration with the Hydrogen Epoch of Reionization Array
• A new MWA limit on the 21 cm Power Spectrum at Redshifts $\sim$ 13 $-$ 17
• Comparing Foreground Removal Techniques for Recovery of the LOFAR-EoR 21cm Power Spectrum
• First upper limits on the 21 cm signal power spectrum from cosmic dawn from one night of observations with NenuFAR
• Optimizing Sparse RFI Prediction using Deep Learning
• Gaussian Process Foreground Subtraction and Power Spectrum Estimation for 21 cm Cosmology
• The FHD/$\mathbf{\epsilon }$ppsilon Epoch of Reionization Power Spectrum Pipeline
• Gaussian Process Regression for foreground removal in HI intensity mapping experiments
• Precision requirements for interferometric gridding in 21-cm power spectrum analysis
• Quantifying Excess Power from Radio Frequency Interference in Epoch of Reionization Measurements
• New EoR Power Spectrum Limits From MWA Phase II Using the Delay Spectrum Method and Novel Systematic Rejection
• Deeper multi-redshift upper limits on the Epoch of Reionization 21-cm signal power spectrum from LOFAR between z=8.3 and z=10.1
• Validation of the HERA Phase I Epoch of Reionization 21 cm Power Spectrum Software Pipeline
• Assessing the impact of two independent direction-dependent calibration algorithms on the LOFAR 21-cm signal power spectrum
• Statistical recovery of 21cm visibilities and their power spectra with Gaussian constrained realisations and Gibbs sampling
• Statistical analysis of the causes of excess variance in the 21 cm signal power spectra obtained with the Low-Frequency Array
• Evidence of Ultra-faint Radio Frequency Interference in Deep 21~cm Epoch of Reionization Power Spectra with the Murchison Widefield Array
• Exploring the cosmic dawn and epoch of reionization with 21cm line
• Statistical recovery of the BAO scale from multipoles of the beam-convolved 21cm correlation function
• The Role of the Instrumental Response in 21 cm EoR Power Spectrum Gridding Analyses
• Constraints on the state of the IGM at $z\sim 8-10$ using redshifted 21-cm observations with LOFAR
• Exploring the consequences of chromatic data excision in 21-cm Epoch of Reionization power spectrum observations
• Measurement of the Anisotropy Power Spectrum of the Radio Synchrotron Background
• SKA-Low Intensity Mapping Pathfinder Updates: Deeper 21 cm Power Spectrum Limits from Improved Analysis Frameworks
• Transient RFI environment of LOFAR-LBA at 72-75 MHz: Impact on ultra-widefield AARTFAAC Cosmic Explorer observations of the redshifted 21-cm signal
• Characterization Of Inpaint Residuals In Interferometric Measurements of the Epoch Of Reionization
• Spectral modelling of Cygnus A between 110 and 250 MHz. Impact on the LOFAR 21-cm signal power spectrum
• Ionospheric contributions to the excess power in high-redshift 21-cm power-spectrum observations with LOFAR
• Why and When to Expect Gaussian Error Distributions in Epoch of Reionization 21-cm Power Spectrum Measurements
• Diffuse Sources, Clustering and the Excess Anisotropy of the Radio Synchrotron Background
• First upper limits on the 21-cm signal power spectrum of neutral hydrogen at $z=9.16$ from the LOFAR 3C196 field
• A Comparative Analysis To Deal With Missing Spectral Information Caused By RFI In Cosmological HI 21CM Observations
• Sensitivity of Bayesian 21 cm power spectrum estimation to foreground model errors
• Aliasing from Galactic Plane Setting in Widefield Radio Interferometry
• The FHD Polarized Imaging Pipeline: A New Approach to Widefield Interferometric Polarimetry
• 21 cm Intensity Mapping with the DSA-2000
• Synthetic Observations with the Square Kilometre Array (SKA) -- development towards an end-to-end pipeline
• TABASCAL: Removing multi-satellite interference from radio interferometry observations
• Epoch of Reionization Power Spectrum Limits from Murchison Widefield Array Data Targeted at EoR1 Field
• The impact of lunar topography on the 21-cm power spectrum for grid-based arrays : Insights for the Dark-ages EXplorer (DEX)
• An SBC-based controller and processor for the laboratory model of PRATUSH Digital Receiver
• Mitigating incoherent excess variance in high-redshift 21 cm observations with multi-output cross-Gaussian process regression