Machine Learning for the Zwicky Transient Facility

Abstract

The Zwicky Transient Facility is a large optical survey in multiple filters producing hundreds of thousands of transient alerts per night. We describe here various machine learning (ML) implementations and plans to make the maximal use of the large data set by taking advantage of the temporal nature of the data, and further combining it with other data sets. We start with the initial steps of separating bogus candidates from real ones, separating stars and galaxies, and go on to the classification of real objects into various classes. Besides the usual methods (e.g., based on features extracted from light curves) we also describe early plans for alternate methods including the use of domain adaptation, and deep learning. In a similar fashion we describe efforts to detect fast moving asteroids. We also describe the use of the Zooniverse platform for helping with classifications through the creation of training samples, and active learning. Finally we mention the synergistic aspects of ZTF and LSST from the ML perspective.

Published in PASP Focus Issue on the Zwicky Transient Facility (doi: 10.1088/1538-3873/aaf3fa). 14 Pages, 8 Figures

References in corpus (52)

• Scikit-learn: Machine Learning in Python
• XGBoost: A Scalable Tree Boosting System
• Gaia Data Release 2. Summary of the contents and survey properties
• Deep Residual Learning for Image Recognition
• Gaia Data Release 1. Summary of the astrometric, photometric, and survey properties
• The Zwicky Transient Facility: System Overview, Performance, and First Results
• The Man Behind the Curtain: X-rays Drive the UV through NIR Variability in the 2013 AGN Outburst in NGC 2617
• Galaxy Zoo : Morphologies derived from visual inspection of galaxies from the Sloan Digital Sky Survey
• First Results from the Catalina Real-time Transient Survey
• The Zwicky Transient Facility: Data Processing, Products, and Archive
• ATLAS: A High-Cadence All-Sky Survey System
• The Palomar Transient Factory: System Overview, Performance and First Results
• A box-fitting algorithm in the search for periodic transits
• The Pan-STARRS1 Surveys
• The Catalina Surveys Periodic Variable Star Catalog
• Weak Gravitational Lensing with COSMOS: Galaxy Selection and Shape Measurements
• The ASAS-SN Catalog of Variable Stars I: The Serendipitous Survey
• A First Catalog of Variable Stars Measured by the Asteroid Terrestrial-impact Last Alert System (ATLAS)
• Xception: Deep Learning with Depthwise Separable Convolutions
• On Machine-Learned Classification of Variable Stars with Sparse and Noisy Time-Series Data
• Proper image subtraction - optimal transient detection, photometry and hypothesis testing
• Automated Transient Identification in the Dark Energy Survey
• The Zwicky Transient Facility Alert Distribution System
• A recurrent neural network for classification of unevenly sampled variable stars
• Using Machine Learning for Discovery in Synoptic Survey Imaging
• Deep Recurrent Neural Networks for Supernovae Classification
• A Morphological Classification Model to Identify Unresolved PanSTARRS1 Sources: Application in the ZTF Real-Time Pipeline
• Comparative performance of selected variability detection techniques in photometric time series
• Machine Learning-based Brokers for Real-time Classification of the LSST Alert Stream
• Asteroid lightcurves from the Palomar Transient Factory survey: Rotation periods and phase functions from sparse photometry
• Gaia Data Release 2: The first Gaia catalogue of long-period variable candidates
• A Package for the Automated Classification of Periodic Variable Stars
• Optimizing spectroscopic follow-up strategies for supernova photometric classification with active learning
• Machine learning for transient discovery in Pan-STARRS1 difference imaging
• Using conditional entropy to identify periodicity
• How to Find More Supernovae with Less Work: Object Classification Techniques for Difference Imaging
• Active Learning to Overcome Sample Selection Bias: Application to Photometric Variable Star Classification
• Improved Asteroid Astrometry and Photometry with Trail Fitting
• Machine learning search for variable stars
• Effective Image Differencing with ConvNets for Real-time Transient Hunting
• The Palomar-Quest Digital Synoptic Sky Survey
• Small near-Earth asteroids in the Palomar Transient Factory survey: A real-time streak-detection system
• Asteroid Spin-Rate Study using the Intermediate Palomar Transient Factory
• Volumetric Survey Speed: A Figure of Merit for Transient Surveys
• FATS: Feature Analysis for Time Series
• 313 new asteroid rotation periods from Palomar Transient Factory observations
• Preparing for advanced LIGO: A Star-Galaxy Separation Catalog for the Palomar Transient Factory
• Towards a Real-time Transient Classification Engine
• Modeling Light Curves for Improved Classification
• ANTARES: A Prototype Transient Broker System
• RATIR Followup of LIGO/Virgo Gravitational Wave Events
• FITS Data Source for Apache Spark

Cited by in corpus (79)

• The Zwicky Transient Facility: Science Objectives
• Seventeen Tidal Disruption Events from the First Half of ZTF Survey Observations: Entering a New Era of Population Studies
• Design and operation of the ATLAS Transient Science Server
• The Zwicky Transient Facility: Observing System
• The Zwicky Transient Facility Catalog of Periodic Variable Stars
• The Zwicky Transient Facility Bright Transient Survey. II. A Public Statistical Sample for Exploring Supernova Demographics
• A tidal disruption event coincident with a high-energy neutrino
• Real-bogus classification for the Zwicky Transient Facility using deep learning
• Discovery and confirmation of the shortest gamma ray burst from a collapsar
• Surveying the reach and maturity of machine learning and artificial intelligence in astronomy
• GROWTH on S190425z: Searching thousands of square degrees to identify an optical or infrared counterpart to a binary neutron star merger with the Zwicky Transient Facility and Palomar Gattini IR
• The Koala: A Fast Blue Optical Transient with Luminous Radio Emission from a Starburst Dwarf Galaxy at $z=0.27$
• The first tidal disruption flare in ZTF: from photometric selection to multi-wavelength characterization
• The Zwicky Transient Facility Census of the Local Universe I: Systematic search for Calcium rich gap transients reveal three related spectroscopic sub-classes
• A New Class of Changing-Look LINERs
• Fink, a new generation of broker for the LSST community
• Kilonova Luminosity Function Constraints based on Zwicky Transient Facility Searches for 13 Neutron Star Mergers
• A Search for Extragalactic Fast Blue Optical Transients in ZTF and the Rate of AT2018cow-like Transients
• Transient processing and analysis using $\text{AMPEL}$: Alert Management, Photometry and Evaluation of Lightcurves
• A data science platform to enable time-domain astronomy
• The Type Icn SN 2021csp: Implications for the Origins of the Fastest Supernovae and the Fates of Wolf-Rayet Stars
• The LSST DESC DC2 Simulated Sky Survey
• Evidence for Late-stage Eruptive Mass-loss in the Progenitor to SN2018gep, a Broad-lined Ic Supernova: Pre-explosion Emission and a Rapidly Rising Luminous Transient
• Applications and Techniques for Fast Machine Learning in Science
• Palomar Gattini-IR: Survey overview, data processing system, on-sky performance and first results
• Detection and Classification of Astronomical Targets with Deep Neural Networks in Wide Field Small Aperture Telescopes
• Alert Classification for the ALeRCE Broker System: The Real-time Stamp Classifier
• SN2020bvc: a Broad-lined Type Ic Supernova with a Double-peaked Optical Light Curve and a Luminous X-ray and Radio Counterpart
• The Spectacular Ultraviolet Flash From the Peculiar Type Ia Supernova 2019yvq
• Classifying the unknown: discovering novel gravitational-wave detector glitches using similarity learning
• Transient-optimised real-bogus classification with Bayesian Convolutional Neural Networks -- sifting the GOTO candidate stream
• Cosmological Fast Optical Transients with the Zwicky Transient Facility: A Search for Dirty Fireballs
• 2900 square degree search for the optical counterpart of short gamma-ray burst GRB 180523B with the Zwicky Transient Facility
• SNIa-Cosmology Analysis Results from Simulated LSST Images: from Difference Imaging to Constraints on Dark Energy
• Detection, Instance Segmentation, and Classification for Astronomical Surveys with Deep Learning (DeepDISC): Detectron2 Implementation and Demonstration with Hyper Suprime-Cam Data
• Four (Super)luminous Supernovae from the First Months of the ZTF Survey
• Discovery of an intermediate-luminosity red transient in M51 and its likely dust-obscured, infrared-variable progenitor
• Fink: early supernovae Ia classification using active learning
• AGN on the move: A search for off-nuclear AGN from recoiling SMBHs and ongoing galaxy mergers with the Zwicky Transient Facility
• An Infrared Search for Kilonovae with the WINTER Telescope. I. Binary Neutron Star Mergers
• Machine Learning applied to asteroid dynamics
• Collapsars as Sites of r-process Nucleosynthesis: Systematic Near-Infrared Follow-up of Type Ic-BL Supernovae
• SUPPNet: Neural network for stellar spectrum normalisation
• Optimizing Serendipitous Detections of Kilonovae: Cadence and Filter Selection
• Vetting the optical transient candidates detected by the GWAC network using convolutional neural networks
• Spectroscopic Studies of Type Ia Supernovae Using LSTM Neural Networks
• Towards Efficient Detection of Small Near-Earth Asteroids Using the Zwicky Transient Facility (ZTF)
• Photometric identification of compact galaxies, stars and quasars using multiple neural networks
• The Broad-lined Ic Supernova ZTF18aaqjovh (SN 2018bvw): An Optically-discovered Engine-driven Supernova Candidate with Luminous Radio Emission
• $\text{tdescore}$: An Accurate Photometric Classifier for Tidal Disruption Events
• What's the Difference? The potential for Convolutional Neural Networks for transient detection without template subtraction
• Radio Afterglows from Compact Binary Coalescences: Prospects for Next-Generation Telescopes
• The effect of phased recurrent units in the classification of multiple catalogs of astronomical lightcurves
• Enabling the discovery of fast transients: A kilonova science module for the Fink broker
• The mini-GWAC optical follow-up of the gravitational wave alerts: results from the O2 campaign and prospects for the upcoming O3 run
• Synthetic tracking using ZTF Long Dwell Datasets
• Enhanced Astronomical Source Classification with Integration of Attention Mechanisms and Vision Transformers
• In search of short gamma-ray burst optical counterpart with the Zwicky Transient Facility
• Machine Learning based search for Cataclysmic Variables within Gaia Science Alerts
• The Evryscope Fast Transient Engine: Real-Time Detection for Rapidly Evolving Transients
• Deep Learning and LLM-based Methods Applied to Stellar Lightcurve Classification
• Real-time Discovery of AT2020xnd: A Fast, Luminous Ultraviolet Transient with Minimal Radioactive Ejecta
• Machine learning technique for morphological classification of galaxies from SDSS. II. The image-based morphological catalogs of galaxies at 0.02<z<0.1
• SN 2022joj: A Peculiar Type Ia Supernova Possibly Driven by an Asymmetric Helium-shell Double Detonation
• Automated classification of eclipsing binary systems in the VVV Survey
• Deep-learning Real/Bogus classification for the Tomo-e Gozen transient survey
• Identifying Transients in the Dark Energy Survey using Convolutional Neural Networks
• GWOPS: A VO-technology Driven Tool to Search for the Electromagnetic Counterpart of Gravitational Wave Event
• Early Identification of Optical Tidal Disruption Events: A science module for the Fink broker
• PAH Marks the Spot: Digging for Buried Clusters in Nearby Star-forming Galaxies
• The automation of optical transient discovery and classification in Rubin-era time-domain astronomy
• Transient electromagnetic sources can detect solitary black holes in Milky Way galaxy
• The Galaxy Activity, Torus, and Outflow Survey (GATOS). Black hole mass estimation using machine learning
• A Morphological Model to Separate Resolved-Unresolved Sources in the DESI Legacy Surveys: Application in the LS4 Alert Stream
• The Hubble Image Similarity Project
• Identifications of RR Lyrae stars and Quasars from the simulated data of Mephisto-W Survey
• Exploring Late Stellar Evolution in the Era of Large Surveys: Machine Learning Prospects for Hot Subdwarfs and White Dwarfs
• Smart obervation method with wide field small aperture telescopes for real time transient detection
• The Large Array Survey Telescope-Pipeline. II. Image Subtraction and Transient Detection