Astronomaly: Personalised Active Anomaly Detection in Astronomical Data

Abstract

Survey telescopes such as the Vera C. Rubin Observatory and the Square Kilometre Array will discover billions of static and dynamic astronomical sources. Properly mined, these enormous datasets will likely be wellsprings of rare or unknown astrophysical phenomena. The challenge is that the datasets are so large that most data will never be seen by human eyes; currently the most robust instrument we have to detect relevant anomalies. Machine learning is a useful tool for anomaly detection in this regime. However, it struggles to distinguish between interesting anomalies and irrelevant data such as instrumental artefacts or rare astronomical sources that are simply not of interest to a particular scientist. Active learning combines the flexibility and intuition of the human brain with the raw processing power of machine learning. By strategically choosing specific objects for expert labelling, it minimises the amount of data that scientists have to look through while maximising potential scientific return. Here we introduce Astronomaly: a general anomaly detection framework with a novel active learning approach designed to provide personalised recommendations. Astronomaly can operate on most types of astronomical data, including images, light curves and spectra. We use the Galaxy Zoo dataset to demonstrate the effectiveness of Astronomaly, as well as simulated data to thoroughly test our new active learning approach. We find that for both datasets, Astronomaly roughly doubles the number of interesting anomalies found in the first 100 objects viewed by the user. Astronomaly is easily extendable to include new feature extraction techniques, anomaly detection algorithms and even different active learning approaches. The code is publicly available at https://github.com/MichelleLochner/astronomaly.

arXiv comment26 pages, 11 figures. Matches published version. 2 minute video introducing Astronomaly: https://youtu.be/vyU0gJFiFxA

References in corpus (23)

• SciPy 1.0--Fundamental Algorithms for Scientific Computing in Python
• Astropy: A Community Python Package for Astronomy
• The Astropy Project: Building an inclusive, open-science project and status of the v2.0 core package
• LSST: from Science Drivers to Reference Design and Anticipated Data Products
• The Zwicky Transient Facility: System Overview, Performance, and First Results
• Galaxy Zoo : Morphologies derived from visual inspection of galaxies from the Sloan Digital Sky Survey
• The Zwicky Transient Facility: Science Objectives
• Galaxy Zoo 1 : Data Release of Morphological Classifications for nearly 900,000 galaxies
• Galaxy Zoo 2: detailed morphological classifications for 304,122 galaxies from the Sloan Digital Sky Survey
• Galaxy Zoo Green Peas: Discovery of A Class of Compact Extremely Star-Forming Galaxies
• Galaxy Zoo: The properties of merging galaxies in the nearby Universe - local environments, colours, masses, star-formation rates and AGN activity
• Galaxy Zoo: the fraction of merging galaxies in the SDSS and their morphologies
• Galaxy Zoo: Probabilistic Morphology through Bayesian CNNs and Active Learning
• The weirdest SDSS galaxies: results from an outlier detection algorithm
• An automatic taxonomy of galaxy morphology using unsupervised machine learning
• Identifying galaxies, quasars, and stars with machine learning: A new catalogue of classifications for 111 million SDSS sources without spectra
• Active Learning to Overcome Sample Selection Bias: Application to Photometric Variable Star Classification
• Detecting outliers in astronomical images with deep generative networks
• Systematic Serendipity: A Test of Unsupervised Machine Learning as a Method for Anomaly Detection
• Unsupervised machine learning for transient discovery in Deeper, Wider, Faster light curves
• Automated novelty detection in the WISE survey with one-class support vector machines
• A semi-supervised Machine Learning search for never-seen Gravitational-Wave sources
• A Flexible Framework for Anomaly Detection via Dimensionality Reduction

Cited by in corpus (44)

• Galaxy Zoo DECaLS: Detailed Visual Morphology Measurements from Volunteers and Deep Learning for 314,000 Galaxies
• The MeerKAT Galaxy Cluster Legacy Survey I. Survey Overview and Highlights
• The DAWES review 10: The impact of deep learning for the analysis of galaxy surveys
• Practical Galaxy Morphology Tools from Deep Supervised Representation Learning
• Searching for changing-state AGNs in massive datasets -- I: applying deep learning and anomaly detection techniques to find AGNs with anomalous variability behaviours
• A review of unsupervised learning in astronomy
• Anomaly detection in Hyper Suprime-Cam galaxy images with generative adversarial networks
• A unique, ring-like radio source with quadrilateral structure detected with machine learning
• Learning to detect RFI in radio astronomy without seeing it
• Constraining the giant radio galaxy population with machine learning and Bayesian inference
• Autoencoding Galaxy Spectra I: Architecture
• Real-Time Detection of Anomalies in Large-Scale Transient Surveys
• A method for finding anomalous astronomical light curves and their analogs
• Preparing to discover the unknown with Rubin LSST -- I: Time domain
• Astronomaly at scale: searching for anomalies amongst 4 million galaxies
• B/PS bulges in DESI Legacy edge-on galaxies I: Sample building
• The ROAD to discovery: machine learning-driven anomaly detection in radio astronomy spectrograms
• Enabling Unsupervised Discovery in Astronomical Images through Self-Supervised Representations
• Identifying outliers in astronomical images with unsupervised machine learning
• Supernova search with active learning in ZTF DR3
• Wide Area VISTA Extra-galactic Survey (WAVES): Unsupervised star-galaxy separation on the WAVES-Wide photometric input catalogue using UMAP and $\mathrm{HDBSCAN}$
• Bursts from Space: MeerKAT - The first citizen science project dedicated to commensal radio transients
• Identifying anomalous radio sources in the EMU Pilot Survey using a complexity-based approach
• Anomaly Detection and Radio-frequency Interference Classification with Unsupervised Learning in Narrowband Radio Technosignature Searches
• SNAD Transient Miner: Finding Missed Transient Events in ZTF DR4 using k-D trees
• Finding AGN remnant candidates based on radio morphology with machine learning
• A Classifier-Based Approach to Multi-Class Anomaly Detection for Astronomical Transients
• A Census of Protostellar Outflows in Nearby Molecular Clouds
• Self-supervised contrastive learning of radio data for source detection, classification and peculiar object discovery
• Astro-COLIBRI 2 -- an advanced platform for real-time multi-messenger discoveries
• The weird and the wonderful in our Solar System: Searching for serendipity in the Legacy Survey of Space and Time
• Analysis of Ring Galaxies Detected Using Deep Learning with Real and Simulated Data
• A Data-Driven Search For Mid-Infrared Excesses Among Five Million Main-Sequence FGK Stars
• Finding radio transients with anomaly detection and active learning based on volunteer classifications
• How to set up your first machine learning project in astronomy
• Real-Time Active Learning for optimised spectroscopic follow-up: Enhancing early SN Ia classification with the Fink broker
• Learning to Detect Interesting Anomalies
• Interpreting anomaly detection of SDSS spectra
• Identifying Astrophysical Anomalies in 99.6 Million Cutouts from the Hubble Legacy Archive Using AnomalyMatch
• AnomalyMatch: Discovering Rare Objects of Interest with Semi-supervised and Active Learning
• Identifying Anomalous DESI Galaxy Spectra with a Variational Autoencoder
• Prospecting MeerKAT Continuum Data for Enigmatic Radio Sources with Unsupervised Vector-Quantised Variational Autoencoders
• Enabling New Discoveries with Machine Learning
• J-PLUS: Understanding outlier white dwarfs in the third data release via dimensionality reduction