IQRM: real-time adaptive RFI masking for radio transient and pulsar searches

Abstract

In a search for short timescale astrophysical transients in time-domain data, radio-frequency interference (RFI) causes both large quantities of false positive candidates and a significant reduction in sensitivity if not correctly mitigated. Here we propose an algorithm that infers a time-variable frequency channel mask directly from short-duration ($\sim$1 s) data blocks: the method consists of computing a spectral statistic that correlates well with the presence of RFI, and then finding high outliers among the resulting values. For the latter task, we propose an outlier detection algorithm called Inter-Quartile Range Mitigation (IQRM), that is both non-parametric and robust to the presence of a trend in sequential data. The method requires no training and can in principle adapt to any telescope and RFI environment; its efficiency is shown on data from both the MeerKAT and Lovell 76-m radio telescopes. IQRM is fast enough to be used in a streaming search and has been integrated into the MeerTRAP real-time transient search pipeline. Open-source Python and C++ implementations are also provided.

Accepted for publication in MNRAS, 11 pages, 9 figures. Minor corrections, improved clarity, additional analysis of impact on single pulse searches. IQRM implementations can be found at https://github.com/v-morello/iqrm (Python) and https://gitlab.com/kmrajwade/iqrm_apollo (C++)

References in corpus (7)

• Post-correlation radio frequency interference classification methods
• Possible periodic activity in the repeating FRB 121102
• Optimal periodicity searching: Revisiting the Fast Folding Algorithm for large-scale pulsar surveys
• An Interference Removal Technique for Radio Pulsar Searches
• Fourier domain excision of periodic radio frequency interference
• Single Pulse Detection Algorithms for Real-time Fast Radio Burst Searches using GPUs
• Multidimensional RFI Framework for Characterising Radio Astronomy Observatories

Cited by in corpus (24)

• The second data release from the European Pulsar Timing Array I. The dataset and timing analysis
• Evidence for an abundant old population of Galactic ultra long period magnetars and implications for fast radio bursts
• The MPIfR-MeerKAT Galactic Plane survey I -- System setup and early results
• The FAST Galactic Plane Pulsar Snapshot survey: VI. The discovery of 473 new pulsars
• TRAPUM discovery of thirteen new pulsars in NGC 1851 using MeerKAT
• Discovery and timing of ten new millisecond pulsars in the globular cluster Terzan 5
• Four pulsar discoveries in NGC 6624 by TRAPUM using MeerKAT
• A sample of Fast Radio Bursts discovered and localised with MeerTRAP at the MeerKAT telescope
• PulsarX: a new pulsar searching package -I. A high performance folding program for pulsar surveys
• The Southern-sky MWA Rapid Two-metre (SMART) pulsar survey -- I. Survey design and processing pipeline
• Missing for 20 years: MeerKAT re-detects the elusive binary pulsar M30B
• The ROAD to discovery: machine learning-driven anomaly detection in radio astronomy spectrograms
• The CRAFT Coherent (CRACO) upgrade I: System Description and Results of the 110-ms Radio Transient Pilot Survey
• Simulating Spectral Kurtosis Mitigation Against Realistic RFI Signals
• The TRAPUM Small Magellanic Cloud pulsar survey with MeerKAT: I. Discovery of seven new pulsars and two Pulsar Wind Nebula associations
• A MeerKAT view of the pulsars in the globular cluster NGC 6522
• An activity transition in FRB 20201124A: methodological rigor, detection of frequency-dependent cessation, and a geometric magnetar model
• Bits missing: Finding exotic pulsars using bfloat16 on NVIDIA GPUs
• Accelerating Dedispersion using Many-Core Architectures
• TRAPUM upper limits on pulsed radio emission for SMC X-ray pulsar J0058-7218
• A renewed search for radio emission from the variable $\gamma$-ray pulsar PSR J2021$+$4026
• A Novel Greedy Approach To Harmonic Summing Using GPUs
• TRAPUM pulsar and transient search in the Sextans A and B galaxies and discovery of background FRB 20210924D
• The Northern Cross Fast Radio Burst project: V. Search for transient radio emission from Galactic magnetars