A novel single-pulse search approach to detection of dispersed radio pulses using clustering and supervised machine learning

Abstract

We present a novel two-stage approach which combines unsupervised and supervised machine learning to automatically identify and classify single pulses in radio pulsar search data. In the first stage, we identify astrophysical pulse candidates in the data, which were derived from the Pulsar Arecibo L-Band Feed Array (PALFA) survey and contain 47,042 independent beams, as trial single-pulse event groups (SPEGs) by clustering single-pulse events and merging clusters that fall within the expected DM and time span of astrophysical pulses. We also present a new peak scoring algorithm, to identify astrophysical peaks in S/N versus DM curves. Furthermore, we group SPEGs detected at a consistent DM for they were likely emitted by the same source. In the second stage, we create a fully labelled benchmark data set by selecting a subset of data with SPEGs identified (using stage 1 procedures), their features extracted and individual SPEGs manually labelled, and then train classifiers using supervised machine learning. Next, using the best trained classifier, we automatically classify unlabelled SPEGs identified in the full data set. To aid the examination of dim SPEGs, we develop an algorithm that searches for an underlying periodicity among grouped SPEGs. The results showed that RandomForest with SMOTE treatment was the best learner, with a recall of 95.6% and a false positive rate of 2.0%. In total, besides all 60 known pulsars from the benchmark data set, the model found 32 additional (i.e., not included in the benchmark data set) known pulsars, and several potential discoveries.

arXiv comment22 pages, accepted for publication in MNRAS, ref. MN-17-3830-MJ.R2

References in corpus (17)

• Scikit-learn: Machine Learning in Python
• SMOTE: Synthetic Minority Over-sampling Technique
• The ATNF Pulsar Catalogue
• A bright millisecond radio burst of extragalactic origin
• The ATNF Pulsar Catalogue
• Transient radio bursts from rotating neutron stars
• Fast Radio Burst Discovered in the Arecibo Pulsar ALFA Survey
• Searches for Fast Radio Transients
• Arecibo Pulsar Survey Using ALFA. I. Survey Strategy and First Discoveries
• PALFA Discovery of a Highly Relativistic Double Neutron Star Binary
• Arecibo Pulsar Survey Using ALFA: Probing Radio Pulsar Intermittency and Transients
• The Millisecond Radio Sky: Transients from a Blind Single Pulse Search
• Arecibo Pulsar Survey Using ALFA. IV. Mock Spectrometer Data Analysis, Survey Sensitivity, and the Discovery of 41 Pulsars
• Discovery of 28 pulsars using new techniques for sorting pulsar candidates
• Discovery and Follow-up of Rotating Radio Transients with the Green Bank and LOFAR Telescopes
• Detection of Dispersed Radio Pulses: A machine learning approach to candidate identification and classification
• Arecibo Pulsar Survey Using ALFA. III. Precursor Survey and Population Synthesis

Cited by in corpus (11)

• Fast Radio Bursts
• Surveying the reach and maturity of machine learning and artificial intelligence in astronomy
• A review of unsupervised learning in astronomy
• The second set of pulsar discoveries by CHIME/FRB/Pulsar: 14 Rotating Radio Transients and 7 pulsars
• CHIME/FRB/Pulsar discovery of a nearby long period radio transient with a timing glitch
• A GPU based single-pulse search pipeline (GSP) with database and its application to the commensal radio astronomy FAST survey (CRAFTS)
• Robust Assessment of Clustering Methods for Fast Radio Transient Candidates
• Vela Pulsar: Single Pulses Analysis with Machine Learning Techniques
• Detecting Pulsars with Neural Networks: A Proof of Concept
• Scalable Solutions for Automated Single Pulse Identification and Classification in Radio Astronomy
• Detecting FRB by DANCE: a method based on DEnsity ANalysis and Cluster Extraction