A Framework for HI Spectral Source Finding Using Distributed-Memory Supercomputing

Abstract

The latest generation of radio astronomy interferometers will conduct all sky surveys with data products consisting of petabytes of spectral line data. Traditional approaches to identifying and parameterising the astrophysical sources within this data will not scale to datasets of this magnitude, since the performance of workstations will not keep up with the real-time generation of data. For this reason, it is necessary to employ high performance computing systems consisting of a large number of processors connected by a high-bandwidth network. In order to make use of such supercomputers substantial modifications must be made to serial source finding code. To ease the transition, this work presents the Scalable Source Finder Framework, a framework providing storage access, networking communication and data composition functionality, which can support a wide range of source finding algorithms provided they can be applied to subsets of the entire image. Additionally, the Parallel Gaussian Source Finder was implemented using SSoFF, utilising Gaussian filters, thresholding, and local statistics. PGSF was able to search on a 256GB simulated dataset in under 24 minutes, significantly less than the 8 to 12 hour observation that would generate such a dataset.

arXiv comment15 pages, 6 figures

References in corpus (8)

• The HIPASS Catalogue - I. Data Presentation
• The Atlas3D project -- XIII. Mass and morphology of HI in early-type galaxies as a function of environment
• Duchamp: a 3D source finder for spectral-line data
• Source-finding for the Australian Square Kilometre Array Pathfinder
• Comparison of potential ASKAP HI survey source finders
• The Characterised Noise Hi source finder: Detecting Hi galaxies using a novel implementation of matched filtering
• 2D-1D Wavelet Reconstruction As A Tool For Source Finding In Spectroscopic Imaging Surveys
• Assessing The Accuracy Of Radio Astronomy Source Finding Algorithms

Cited by in corpus (2)

• SKA Science Data Challenge 2: analysis and results
• Performance Analysis of GPU-Accelerated Filter-Based Source Finding for HI Spectral Line Image Data