Anomaly detection in the Zwicky Transient Facility DR3

Abstract

We present results from applying the SNAD anomaly detection pipeline to the third public data release of the Zwicky Transient Facility (ZTF DR3). The pipeline is composed of 3 stages: feature extraction, search of outliers with machine learning algorithms and anomaly identification with followup by human experts. Our analysis concentrates in three ZTF fields, comprising more than 2.25 million objects. A set of 4 automatic learning algorithms was used to identify 277 outliers, which were subsequently scrutinised by an expert. From these, 188 (68%) were found to be bogus light curves -- including effects from the image subtraction pipeline as well as overlapping between a star and a known asteroid, 66 (24%) were previously reported sources whereas 23 (8%) correspond to non-catalogued objects, with the two latter cases of potential scientific interest (e. g. 1 spectroscopically confirmed RS Canum Venaticorum star, 4 supernovae candidates, 1 red dwarf flare). Moreover, using results from the expert analysis, we were able to identify a simple bi-dimensional relation which can be used to aid filtering potentially bogus light curves in future studies. We provide a complete list of objects with potential scientific application so they can be further scrutinised by the community. These results confirm the importance of combining automatic machine learning algorithms with domain knowledge in the construction of recommendation systems for astronomy. Our code is publicly available at https://github.com/snad-space/zwad

31 pages, 28 figures, 4 tables, accepted for publication in MNRAS

References in corpus (62)

• Scikit-learn: Machine Learning in Python
• SciPy 1.0--Fundamental Algorithms for Scientific Computing in Python
• Astropy: A Community Python Package for Astronomy
• The NumPy array: a structure for efficient numerical computation
• Gaia Data Release 2. Summary of the contents and survey properties
• The Wide-field Infrared Survey Explorer (WISE): Mission Description and Initial On-orbit Performance
• The Astropy Project: Building an inclusive, open-science project and status of the v2.0 core package
• Measuring Reddening with SDSS Stellar Spectra and Recalibrating SFD
• The Transiting Exoplanet Survey Satellite
• The SIMBAD astronomical database
• The Zwicky Transient Facility: System Overview, Performance, and First Results
• Estimating distances from parallaxes IV: Distances to 1.33 billion stars in Gaia Data Release 2
• Sloan Digital Sky Survey IV: Mapping the Milky Way, Nearby Galaxies, and the Distant Universe
• The Sixteenth Data Release of the Sloan Digital Sky Surveys: First Release from the APOGEE-2 Southern Survey and Full Release of eBOSS Spectra
• First Results from the Catalina Real-time Transient Survey
• The Zwicky Transient Facility: Data Processing, Products, and Archive
• HyperLEDA. III. The catalogue of extragalactic distances
• SALT2: using distant supernovae to improve the use of Type Ia supernovae as distance indicators
• astroquery: An Astronomical Web-Querying Package in Python
• The Pan-STARRS1 Database and Data Products
• The Indo-U.S. Library of Coude Feed Stellar Spectra
• The Combined Ultraviolet, Optical, and Near-Infrared Light Curves of the Kilonova Associated with the Binary Neutron Star Merger GW170817: Unified Data Set, Analytic Models, and Physical Implications
• Galaxy Zoo Green Peas: Discovery of A Class of Compact Extremely Star-Forming Galaxies
• The Effect of a Strong Stellar Flare on the Atmospheric Chemistry of an Earth-like Planet Orbiting an M dwarf
• An Open Catalog for Supernova Data
• Data Mining and Machine Learning in Astronomy
• A First Catalog of Variable Stars Measured by the Asteroid Terrestrial-impact Last Alert System (ATLAS)
• The Zwicky Transient Facility Catalog of Periodic Variable Stars
• Planet Hunters X. KIC 8462852 - Where's the Flux?
• The Ultraviolet Radiation Environment Around M dwarf Exoplanet Host Stars
• Absolute-Magnitude Distributions of Supernovae
• An all-sky Support Vector Machine selection of WISE YSO Candidates
• The weirdest SDSS galaxies: results from an outlier detection algorithm
• Fink, a new generation of broker for the LSST community
• A New Catalog of HII Regions in M31
• Machine Learning-based Brokers for Real-time Classification of the LSST Alert Stream
• Red Supergiants in the Andromeda Galaxy (M31)
• The EPOCH Project: I. Periodic variable stars in the EROS-2 LMC database
• Reliable Identifications of AGN from the WISE, 2MASS and Rosat all-sky surveys
• Catalogue of new Herbig Ae/Be and classical Be stars. A machine learning approach to Gaia DR2
• The ANTARES Astronomical Time-Domain Event Broker
• Anomaly detection for machine learning redshifts applied to SDSS galaxies
• The ASAS-SN Catalog of Variable Stars VI: An All-Sky Sample of $\delta$ Scuti Stars
• Discovering the Unexpected in Astronomical Survey Data
• An analysis of feature relevance in the classification of astronomical transients with machine learning methods
• Luminous and Variable Stars in M31 and M33. IV. Luminous Blue Variables, Candidate LBVs, and the B[e] Supergiants; How to Tell Them Apart
• Systematic Serendipity: A Test of Unsupervised Machine Learning as a Method for Anomaly Detection
• Active Anomaly Detection for time-domain discoveries
• Anomaly Detection in the Open Supernova Catalog
• Caucasian mountain observatory of Sternberg astronomical institute: six years of operation
• Unsupervised machine learning for transient discovery in Deeper, Wider, Faster light curves
• DIRECT Distances to Nearby Galaxies Using Detached Eclipsing Binaries and Cepheids. IV. Variables in the Field M31D
• Identification of 13 Cepheids and 333 Other Variables in M31
• Properties of M31. II: A Cepheid disk sample derived from the first year of PS1 PAndromeda data
• Automated novelty detection in the WISE survey with one-class support vector machines
• Cepheids in M31 - The PAndromeda Cepheid sample
• Density Based Outlier Scoring on Kepler Data
• A classification algorithm for time-domain novelties in preparation for LSST alerts: Application to variable stars and transients detected with DECam in the Galactic Bulge
• Near-IR spectra of IPHAS extremely red Galactic AGB stars
• Properties of M31. IV: Candidate Luminous Blue Variables from PAndromeda
• Gravitational microlensing events from the first year of the northern Galactic plane survey by the Zwicky Transient Facility
• AT 2020iko: a WZ Sge-type DN candidate with an anomalous precursor event

Cited by in corpus (31)

• The DAWES review 10: The impact of deep learning for the analysis of galaxy surveys
• The Young Supernova Experiment Data Release 1 (YSE DR1): Light Curves and Photometric Classification of 1975 Supernovae
• A Deep Learning Approach for Active Anomaly Detection of Extragalactic Transients
• Searching for changing-state AGNs in massive datasets -- I: applying deep learning and anomaly detection techniques to find AGNs with anomalous variability behaviours
• Active Anomaly Detection for time-domain discoveries
• Detection, Instance Segmentation, and Classification for Astronomical Surveys with Deep Learning (DeepDISC): Detectron2 Implementation and Demonstration with Hyper Suprime-Cam Data
• The SNAD Viewer: Everything You Want to Know about Your Favorite ZTF Object
• Applications of Machine Learning to Predicting Core-collapse Supernova Explosion Outcomes
• Learning to detect RFI in radio astronomy without seeing it
• Real-Time Detection of Anomalies in Large-Scale Transient Surveys
• Deep Attention-Based Supernovae Classification of Multi-Band Light-Curves
• A method for finding anomalous astronomical light curves and their analogs
• Persistent and occasional: searching for the variable population of the ZTF/4MOST sky using ZTF data release 11
• The ROAD to discovery: machine learning-driven anomaly detection in radio astronomy spectrograms
• Machine Learning and Cosmology
• Identifying Light-curve Signals with a Deep Learning Based Object Detection Algorithm. II. A General Light Curve Classification Framework
• Searching for Anomalies in the ZTF Catalog of Periodic Variable Stars
• Transient Classifiers for Fink: Benchmarks for LSST
• Supernova search with active learning in ZTF DR3
• SNAD Transient Miner: Finding Missed Transient Events in ZTF DR4 using k-D trees
• A Classifier-Based Approach to Multi-Class Anomaly Detection for Astronomical Transients
• Perfecting our set of spectrophotometric standard DA white dwarfs
• Peculiarities of the chemical abundance distribution in galaxies NGC 3963 and NGC 7292
• Finding radio transients with anomaly detection and active learning based on volunteer classifications
• Real-bogus scores for active anomaly detection
• Exploring the Universe with SNAD: Anomaly Detection in Astronomy
• Early Detection of Multiwavelength Blazar Variability
• The largest ground-based catalogue of M-dwarf flares
• XMM-Newton follow-up of two eROSITA X-ray binary candidates
• J-PLUS: Understanding outlier white dwarfs in the third data release via dimensionality reduction
• AnomalyMatch: Discovering Rare Objects of Interest with Semi-supervised and Active Learning