SERPent: Automated reduction and RFI-mitigation software for e-MERLIN
arXiv:1401.8181 · doi:10.1016/j.ascom.2013.09.001
Abstract
The Scripted E-merlin Rfi-mitigation PipelinE for iNTerferometry (SERPent) is an automated reduction and RFI-mitigation procedure utilising the SumThreshold methodology (Offringa et al. 2010b), originally developed for the LOFAR pipeline. SERPent is written in the Parseltongue language enabling interaction with the Astronomical Image Processing Software (AIPS) program. Moreover, SERPent is a simple "out of the box" Python script, which is easy to set up and is free of compilers. In addition to the flagging of RFI affected visibilities, the script also flags antenna zero-amplitude dropouts and Lovell telescope phase calibrator stationary scans inherent to the e-MERLIN system. Both the flagging and computational performances of SERPent are presented here, for e-MERLIN commissioning datasets for both L-band (1.3 - 1.8 GHz) and C-band (4 - 8 GHz) observations. RFI typically amounts to < 20 - 25% for the more problematic L-band observations and < 5% for the generally RFI quieter C-band. The level of RFI detection and flagging is more accurate and delicate than visual manual flagging, with the output immediately ready for AIPS calibration. SERPent is fully parallelised and has been tested on a range of computing systems. The current flagging rate is at 110 GB/day on a "high-end" computer (16 CPUs, 100 GB memory) which amounts to ~ 6.9 GB/CPU/day, with an expected increase in performance when e-MERLIN has completed its commissioning. The refining of automated reduction and calibration procedures is essential for the e-MERLIN legacy projects and future interferometers such as the SKA and the associated pathfinders (MeerKAT and ASKAP), where the vast data sizes (> TB) make traditional astronomer interactions unfeasible.
arXiv comment15 pages, 13 figures, Astronomy and Computing, Volume 2, p. 54-66
References in corpus (6)
- Post-correlation radio frequency interference classification methods
- Removing radio interference from contaminated astronomical spectra using an independent reference signal and closure relations
- Radio Frequency Interference Mitigation at the WSRT
- Statistically Stable Estimates of Variance in Radioastronomical Observations as Tools for RFI Mitigation
- A new approach to mitigation of radio frequency interference in interferometric data
- A LOFAR RFI detection pipeline and its first results
Cited by in corpus (21)
- LeMMINGs. I. The eMERLIN legacy survey of nearby galaxies. 1.5-GHz parsec-scale radio structures and cores
- Absolving the SSINS of Precision Interferometric Radio Data: A New Technique for Mitigating Faint Radio Frequency Interference
- The impact of interference excision on 21-cm Epoch of Reionization power spectrum analyses
- Coexistence and Spectrum Sharing Above 100 GHz
- Radio jets in NGC 4151: where eMERLIN meets HST
- LeMMINGs. II. The e-MERLIN legacy survey of nearby galaxies. The deepest radio view of the Palomar sample on parsec scale
- Revisiting the birth locations of pulsars B1929+10, B2020+28,and B2021+51
- The e-MERLIN Galaxy Evolution Survey (e-MERGE): Overview and Survey Description
- Radio Frequency Interference Mitigation based on the ArPLS and SumThreshold Method
- Implementing RFI mitigation in Radio Science
- HIDE & SEEK: End-to-End Packages to Simulate and Process Radio Survey Data
- e-MERLIN 21cm constraints on the mass-loss rates of OB stars in Cyg OB2
- Detecting Radio Frequency Interference in radio-antenna arrays with the Recurrent Neural Network algorithm
- The e-MERLIN Data Reduction Pipeline
- Transient RFI environment of LOFAR-LBA at 72-75 MHz: Impact on ultra-widefield AARTFAAC Cosmic Explorer observations of the redshifted 21-cm signal
- A Radio Census of the Massive Stellar Cluster Westerlund 1
- Radio Frequency Interference Detection Using Efficient Multi-Scale Convolutional Attention UNet
- Unveiling the 100 pc scale nuclear radio structure of NGC 6217 with e-MERLIN and the VLA
- Radio Galaxy Shape Measurement with Hamiltonian Monte Carlo in the Visibility Domain
- An interference detection strategy for Apertif based on AOFlagger 3
- COBRaS: The e-MERLIN 21 cm Legacy survey of Cygnus OB2