Radio Galaxy Zoo: Using semi-supervised learning to leverage large unlabelled data-sets for radio galaxy classification under data-set shift

Abstract

In this work we examine the classification accuracy and robustness of a state-of-the-art semi-supervised learning (SSL) algorithm applied to the morphological classification of radio galaxies. We test if SSL with fewer labels can achieve test accuracies comparable to the supervised state-of-the-art and whether this holds when incorporating previously unseen data. We find that for the radio galaxy classification problem considered, SSL provides additional regularisation and outperforms the baseline test accuracy. However, in contrast to model performance metrics reported on computer science benchmarking data-sets, we find that improvement is limited to a narrow range of label volumes, with performance falling off rapidly at low label volumes. Additionally, we show that SSL does not improve model calibration, regardless of whether classification is improved. Moreover, we find that when different underlying catalogues drawn from the same radio survey are used to provide the labelled and unlabelled data-sets required for SSL, a significant drop in classification performance is observered, highlighting the difficulty of applying SSL techniques under dataset shift. We show that a class-imbalanced unlabelled data pool negatively affects performance through prior probability shift, which we suggest may explain this performance drop, and that using the Frechet Distance between labelled and unlabelled data-sets as a measure of data-set shift can provide a prediction of model performance, but that for typical radio galaxy data-sets with labelled sample volumes of O(1000), the sample variance associated with this technique is high and the technique is in general not sufficiently robust to replace a train-test cycle.

arXiv commentAccepted to MNRAS. 14 pages

References in corpus (36)

• The Seventh Data Release of the Sloan Digital Sky Survey
• On the fundamental dichotomy in the local radio-AGN population: accretion, evolution, and host galaxy properties
• EMU: Evolutionary Map of the Universe
• The Rapid ASKAP Continuum Survey I: Design and First Results
• Radio galaxies and feedback from AGN jets
• Generative Adversarial Networks recover features in astrophysical images of galaxies beyond the deconvolution limit
• Revisiting the Fanaroff-Riley dichotomy and radio-galaxy morphology with the LOFAR Two-Metre Sky Survey (LoTSS)
• Deep ATLAS radio observations of the CDFS-SWIRE field
• A pilot study of the radio-emitting AGN population: the emerging new class of FR0 radio-galaxies
• A simulation-based analytic model of radio galaxies
• Classifying Radio Galaxies with Convolutional Neural Network
• Radio Galaxy Zoo: host galaxies and radio morphologies derived from visual inspection
• The Energetics and Lifetimes of Local Radio Active Galactic Nuclei
• Avocado: Photometric Classification of Astronomical Transients with Gaussian Process Augmentation
• Radio Galaxy Zoo: ClaRAN - A Deep Learning Classifier for Radio Morphologies
• A representative survey of the dynamics and energetics of FRII radio galaxies
• Radio Galaxy Zoo: Compact and extended radio source classification with deep learning
• The nuclear properties and extended morphologies of powerful radio galaxies: the roles of host galaxy and environment
• LOFAR discovery of a 700-kpc remnant radio galaxy at low redshift
• Cataloging the radio-sky with unsupervised machine learning: a new approach for the SKA era
• CNN Architecture Comparison for Radio Galaxy Classification
• Radio Galaxy Zoo: Unsupervised Clustering of Convolutionally Auto-encoded Radio-astronomical Images
• Semi-supervised Learning for Photometric Supernova Classification
• AstroVaDEr: Astronomical Variational Deep Embedder for Unsupervised Morphological Classification of Galaxies and Synthetic Image Generation
• Fanaroff-Riley classification of radio galaxies using group-equivariant convolutional neural networks
• Attention-gating for improved radio galaxy classification
• Transfer learning for radio galaxy classification
• Morphological classification of compact and extended radio galaxies using convolutional neural networks and data augmentation techniques
• Understanding the Fanaroff-Riley radio galaxy classification
• A semi-supervised Machine Learning search for never-seen Gravitational-Wave sources
• Rapid sorting of radio galaxy morphology using Haralick features
• Quantifying Uncertainty in Deep Learning Approaches to Radio Galaxy Classification
• Structured Variational Inference for Simulating Populations of Radio Galaxies
• Data-Efficient Classification of Radio Galaxies
• Ram pressure statistics for bent tail radio galaxies
• RadioGAN - Translations between different radio surveys with generative adversarial networks