The Automatic Learning for the Rapid Classification of Events (ALeRCE) Alert Broker

Abstract

We introduce the Automatic Learning for the Rapid Classification of Events (ALeRCE) broker, an astronomical alert broker designed to provide a rapid and self--consistent classification of large etendue telescope alert streams, such as that provided by the Zwicky Transient Facility (ZTF) and, in the future, the Vera C. Rubin Observatory Legacy Survey of Space and Time (LSST). ALeRCE is a Chilean--led broker run by an interdisciplinary team of astronomers and engineers, working to become intermediaries between survey and follow--up facilities. ALeRCE uses a pipeline which includes the real--time ingestion, aggregation, cross--matching, machine learning (ML) classification, and visualization of the ZTF alert stream. We use two classifiers: a stamp--based classifier, designed for rapid classification, and a light--curve--based classifier, which uses the multi--band flux evolution to achieve a more refined classification. We describe in detail our pipeline, data products, tools and services, which are made public for the community (see \url{https://alerce.science}). Since we began operating our real--time ML classification of the ZTF alert stream in early 2019, we have grown a large community of active users around the globe. We describe our results to date, including the real--time processing of $9.7\times 10^7$ alerts, the stamp classification of $1.9\times 10^7$ objects, the light curve classification of $8.5\times 10^5$ objects, the report of 3088 supernova candidates, and different experiments using LSST-like alert streams. Finally, we discuss the challenges ahead to go from a single-stream of alerts such as ZTF to a multi--stream ecosystem dominated by LSST.

arXiv commentSubmitted to AAS on Jun 29th. Preview for LSST PCW 2020. Comments welcome

References in corpus (124)

• Astropy: A Community Python Package for Astronomy
• The Sloan Digital Sky Survey: Technical Summary
• Gaia Data Release 2. Summary of the contents and survey properties
• The Wide-field Infrared Survey Explorer (WISE): Mission Description and Initial On-orbit Performance
• Advanced Virgo: a 2nd generation interferometric gravitational wave detector
• The Large Area Telescope on the Fermi Gamma-ray Space Telescope Mission
• The Transiting Exoplanet Survey Satellite
• Multi-messenger Observations of a Binary Neutron Star Merger
• The SIMBAD astronomical database
• The Zwicky Transient Facility: System Overview, Performance, and First Results
• Central Masses and Broad-Line Region Sizes of Active Galactic Nuclei. II. A Homogeneous Analysis of a Large Reverberation-Mapping Database
• First Results from the Catalina Real-time Transient Survey
• The Zwicky Transient Facility: Data Processing, Products, and Archive
• ATLAS: A High-Cadence All-Sky Survey System
• Kepler Mission Design, Realized Photometric Performance, and Early Science
• The All-Sky Automated Survey for Supernovae (ASAS-SN) Light Curve Server v1.0
• The Palomar Transient Factory: System Overview, Performance and First Results
• Neutrino emission from the direction of the blazar TXS 0506+056 prior to the IceCube-170922A alert
• The Hyper Suprime-Cam SSP Survey: Overview and Survey Design
• The Dark Energy Camera
• VISTA Variables in the Via Lactea (VVV): The public ESO near-IR variability survey of the Milky Way
• The MACHO Project: Microlensing Results from 5.7 Years of LMC Observations
• Design Concepts for the Cherenkov Telescope Array
• The IceCube Neutrino Observatory: Instrumentation and Online Systems
• Spitzer Survey of the Large Magellanic Cloud, Surveying the Agents of a Galaxy's Evolution (SAGE) I: Overview and Initial Results
• Intermediate-Mass Black Holes
• Catalogue of cataclysmic binaries, low-mass X-ray binaries and related objects (Seventh edition)
• What Drives the Growth of Black Holes?
• SkyMapper and the Southern Sky Survey
• The Catalina Surveys Periodic Variable Star Catalog
• A Continuum of H- to He-Rich Tidal Disruption Candidates With a Preference for E+A Galaxies
• The Discovery of the First "Changing Look" Quasar: New Insights into the Physics & Phenomenology of AGN
• The IceCube Data Acquisition System: Signal Capture, Digitization, and Timestamping
• An Open Catalog for Supernova Data
• Confined Dense Circumstellar Material Surrounding a Regular Type II Supernova: The Unique Flash-Spectroscopy Event of SN 2013fs
• A Wolf-Rayet-like progenitor of supernova SN 2013cu from spectral observations of a wind
• Rapidly-Evolving and Luminous Transients from Pan-STARRS1
• K-corrections and spectral templates of Type Ia supernovae
• Frequency of Solar-Like Systems and of Ice and Gas Giants Beyond the Snow Line from High-Magnification Microlensing Events in 2005-2008
• The 5th edition of the Roma-BZCAT. A short presentation
• SNANA: A Public Software Package for Supernova Analysis
• Detecting Earth-Mass Planets with Gravitational Microlensing
• Observational evidence for intermediate-mass black holes
• Calcium-rich gap transients in the remote outskirts of galaxies
• Automated supervised classification of variable stars I. Methodology
• Photometric Supernova Classification With Machine Learning
• Microlensing and dynamical constraints on primordial black hole dark matter with an extended mass function
• The ASAS-SN Catalog of Variable Stars II: Uniform Classification of 412,000 Known Variables
• SuperNNova: an open-source framework for Bayesian, Neural Network based supernova classification
• Real-bogus classification for the Zwicky Transient Facility using deep learning
• The delay of shock breakout due to circumstellar material seen in most Type II Supernovae
• A Near-IR Spectroscopic Survey of FU Orionis Objects
• Automating Discovery and Classification of Transients and Variable Stars in the Synoptic Survey Era
• RAPID: Early Classification of Explosive Transients using Deep Learning
• Models and Simulations for the Photometric LSST Astronomical Time Series Classification Challenge (PLAsTiCC)
• The Zwicky Transient Facility Alert Distribution System
• Gaia Data Release 2: Variable stars in the colour-absolute magnitude diagram
• Alert Classification for the ALeRCE Broker System: The Light Curve Classifier
• Accurate dynamical mass determination of a classical Cepheid in an eclipsing binary system
• Exploring the Variable Sky with LINEAR. III. Classification of Periodic Light Curves
• The weirdest SDSS galaxies: results from an outlier detection algorithm
• A recurrent neural network for classification of unevenly sampled variable stars
• A new physical interpretation of optical and infrared variability in quasars
• A New Catalogue of Type 1 AGN and its Implication on the AGN Unified Model
• Deep-HiTS: Rotation Invariant Convolutional Neural Network for Transient Detection
• Understanding extreme quasar optical variability with CRTS: II. Changing-state quasars
• RR-Lyrae-type pulsations from a 0.26-solar-mass star in a binary system
• Avocado: Photometric Classification of Astronomical Transients with Gaussian Process Augmentation
• Fink, a new generation of broker for the LSST community
• Deep Recurrent Neural Networks for Supernovae Classification
• Evryscope science: exploring the potential of all-sky gigapixel-scale telescopes
• Transient processing and analysis using $\text{AMPEL}$: Alert Management, Photometry and Evaluation of Lightcurves
• Machine Learning-based Brokers for Real-time Classification of the LSST Alert Stream
• Construction of a Calibrated Probabilistic Classification Catalog: Application to 50k Variable Sources in the All-Sky Automated Survey
• The EPOCH Project: I. Periodic variable stars in the EROS-2 LMC database
• A population of eruptive variable protostars in VVV
• Gaia Data Release 2: The first Gaia catalogue of long-period variable candidates
• A Package for the Automated Classification of Periodic Variable Stars
• Variable stars across the observational HR diagram
• How to Find More Supernovae with Less Work: Object Classification Techniques for Difference Imaging
• Supernova Photometric Classification Pipelines Trained on Spectroscopically Classified Supernovae from the Pan-STARRS1 Medium-Deep Survey
• A simple and robust method for automated photometric classification of supernovae using neural networks
• Redshift-Independent Distances in the NASA/IPAC Extragalactic Database: Methodology, Content and Use of NED-D
• The QUEST-La Silla AGN variability survey: connection between AGN variability and black hole physical properties
• Hierarchical progressive surveys. Multi-resolution HEALPix data structures for astronomical images, catalogues, and 3-dimensional data cubes
• Supernova Nebular Spectroscopy Suggests a Hybrid Envelope-Stripping Mechanism for Massive Stars
• Deep-Learnt Classification of Light Curves
• catsHTM - A tool for fast accessing and cross-matching large astronomical catalogs
• Machine Learning Classification of SDSS Transient Survey Images
• The High Cadence Transient Survey (HiTS) - I. Survey design and supernova shock breakout constraints
• A transient search using combined human and machine classifications
• An analysis of feature relevance in the classification of astronomical transients with machine learning methods
• Deep multi-survey classification of variable stars
• Deep Learning for Image Sequence Classification of Astronomical Events
• Gaia Data Release 2: All-sky classification of high-amplitude pulsating stars
• The period-luminosity relation of red supergiants with Gaia DR2
• Kiso Supernova Survey (KISS): Survey Strategy
• STACCATO: A Novel Solution to Supernova Photometric Classification with Biased Training Sets
• Automatic Classification of Variable Stars in Catalogs with missing data
• Probing the extragalactic fast transient sky at minute timescales with DECam
• Introducing the Search for Intermediate-mass Black-hole In Nearby Galaxy (SIBLING) Survey
• Robust period estimation using mutual information for multi-band light curves in the synoptic survey era
• Comparing Multi-class, Binary and Hierarchical Machine Learning Classification schemes for variable stars
• Gaia RR Lyrae Stars in Nearby Ultra-Faint Dwarf Satellite Galaxies
• Clustering Based Feature Learning on Variable Stars
• Unsupervised Classification of Variable Stars
• Into the Darkness: Classical and Type II Cepheids in the Zona Galactica Incognita
• Deep Neural Network Classifier for Variable Stars with Novelty Detection Capability
• An irregular discrete time series model to identify residuals with autocorrelation in astronomical light curves
• Meta Classification for Variable Stars
• Mode Switching Time Scales in the Classical Variable Stars
• Supervised Ensemble Classification of Kepler Variable Stars
• Automatic Survey-Invariant Variable Star Classification
• OGLE-ing the Magellanic System: RR Lyrae Stars in the Bridge
• The High Cadence Transient Survey (HITS): Compilation and characterization of light-curve catalogs
• Discrete-time autoregressive model for unequally spaced time-series observations
• Modeling Light Curves for Improved Classification
• Chemical evolution of the Galactic bulge as traced by microlensed dwarf and subgiant stars. VII. Lithium
• Uncertain classification of Variable Stars: handling observational GAPS and noise
• Single-epoch supernova classification with deep convolutional neural networks
• Variable classification in the LSST era: Exploring a model for quasi-periodic light curves
• Circumstellar properties of Type Ia supernovae from the helium star donor channel
• OGLE-ing the Magellanic System: Cepheids in the Bridge
• Variable Star Classification Using Multi-View Metric Learning

Cited by in corpus (115)

• Fink, a new generation of broker for the LSST community
• A Search for Extragalactic Fast Blue Optical Transients in ZTF and the Rate of AT2018cow-like Transients
• A data science platform to enable time-domain astronomy
• Applications and Techniques for Fast Machine Learning in Science
• Alert Classification for the ALeRCE Broker System: The Real-time Stamp Classifier
• YSE-PZ: A Transient Survey Management Platform that Empowers the Human-in-the-Loop
• A Deep Learning Approach for Active Anomaly Detection of Extragalactic Transients
• The Young Supernova Experiment Data Release 1 (YSE DR1): Light Curves and Photometric Classification of 1975 Supernovae
• Searching for changing-state AGNs in massive datasets -- I: applying deep learning and anomaly detection techniques to find AGNs with anomalous variability behaviours
• Luminous Supernovae: Unveiling a Population Between Superluminous and Normal Core-collapse Supernovae
• A Transient "Changing-look'' Active Galactic Nucleus Resolved on Month Timescales from First-year Sloan Digital Sky Survey V Data
• Confirming new changing--look AGNs discovered through optical variability using a random-forest based light curve classifier
• A Systematic Study of Ia-CSM Supernovae from the ZTF Bright Transient Survey
• Velocity-resolved Reverberation Mapping of Changing-look Active Galactic Nucleus NGC~4151 During Outburst Stage: Evidence for Kinematics Evolution of Broad-line Region
• The Iron Yield of Normal Type II Supernovae
• The SNAD Viewer: Everything You Want to Know about Your Favorite ZTF Object
• Fink: early supernovae Ia classification using active learning
• Identifying Tidal Disruption Events with an Expansion of the FLEET Machine Learning Algorithm
• An Infrared Search for Kilonovae with the WINTER Telescope. I. Binary Neutron Star Mergers
• AT 2021loi: A Bowen Fluorescence Flare with a Rebrightening Episode, Occurring in a Previously-Known AGN
• Minutes-duration Optical Flares with Supernova Luminosities
• Limits on High-Frequency Gravitational Waves in Planetary Magnetospheres
• Deep Attention-Based Supernovae Classification of Multi-Band Light-Curves
• Improving the selection of changing-look AGNs through multi-wavelength photometric variability
• Inferencing Progenitor and Explosion Properties of Evolving Core-collapse Supernovae from Zwicky Transient Facility Light Curves
• Polarimetry of Hydrogen-Poor Superluminous Supernovae
• The Type 1 and Type 2 AGN dichotomy according to their ZTF optical variability
• Persistent and occasional: searching for the variable population of the ZTF/4MOST sky using ZTF data release 11
• SDSS1335+0728: The awakening of a $\sim 10^6{M}_{\odot }$ black hole
• Multiple Peaks and a Long Precursor in the Type IIn Supernova 2021qqp: An Energetic Explosion in a Complex Circumstellar Environment
• Velocity-resolved Reverberation Mapping of Changing-look Active Galactic Nucleus NGC 4151 during Outburst Stage. II. Four Season Observation Results
• ATAT: Astronomical Transformer for time series And Tabular data
• Photometry and spectroscopy of the Type Icn supernova 2021ckj: The diverse properties of the ejecta and circumstellar matter of Type Icn SNe
• eRASSt J074426.3+291606: prompt accretion disc formation in a 'faint and slow' tidal disruption event
• Linking Extragalactic Transients and their Host Galaxy Properties: Transient Sample, Multi-Wavelength Host Identification, and Database Construction
• Searching for precursor activity of Type IIn Supernovae
• Identification of 3 cataclysmic variables detected by the ART-XC and eROSITA telescopes aboard SRG during the all-sky X-ray survey
• Enabling the discovery of fast transients: A kilonova science module for the Fink broker
• The DEHVILS Survey Overview and Initial Data Release: High-Quality Near-Infrared Type Ia Supernova Light Curves at Low Redshift
• Supernova 2020wnt: An Atypical Superluminous Supernova with a Hidden Central Engine
• TiDES: The 4MOST Time Domain Extragalactic Survey
• The effect of phased recurrent units in the classification of multiple catalogs of astronomical lightcurves
• Environmental Dependence of Type Ia Supernovae in Low-Redshift Galaxy Clusters
• Minute-Cadence Observations of the LAMOST Fields with the TMTS: II. Catalogues of Short-Period Variable Stars from the First Two-Year Surveys
• HEALPix Alchemy: Fast All-Sky Geometry and Image Arithmetic in a Relational Database for Multimessenger Astronomy Brokers
• Identifying Light-curve Signals with a Deep Learning Based Object Detection Algorithm. II. A General Light Curve Classification Framework
• Deep Drilling in the Time Domain with DECam: Survey Characterization
• Nine New Cataclysmic Variable Stars with Negative Superhumps
• The Mini-SiTian Array: first-two-year operation
• DELIGHT: Deep Learning Identification of Galaxy Hosts of Transients using Multi-resolution Images
• Optical Rebrightening of Extragalactic Transients from the Zwicky Transient Facility
• Transient Classifiers for Fink: Benchmarks for LSST
• Luminous Type II Short-Plateau SN 2023ufx: Asymmetric Explosion of a Partially-Stripped Massive Progenitor
• Tilted discs in six poorly studied cataclysmic variables
• Optical Observations of the Nearby Type Ia Supernova 2021hpr
• Understanding of the properties of neural network approaches for transient light curve approximations
• Multiwavelength monitoring of the nucleus in PBC J2333.9-2343: the giant radio galaxy with a blazar-like core
• SAGUARO: Time-domain Infrastructure for the Fourth Gravitational-wave Observing Run and Beyond
• Machine Learning based search for Cataclysmic Variables within Gaia Science Alerts
• Multi-wavelength observations of multiple eruptions of the recurrent nova M31N 2008-12a
• A study in scarlet -- I. Photometric properties of a sample of Intermediate Luminosity Red Transients
• SN 2020uem: A Possible Thermonuclear Explosion within A Dense Circumstellar Medium (II) The Properties of The CSM from Polarimetry and Light Curve Modeling
• SN 2019nyk: A rapidly declining Type II supernova with early interaction signatures
• SN 2020uem: A Possible Thermonuclear Explosion within A Dense Circumstellar Medium (I) The Nature of Type IIn/Ia-CSM SNe from Photometry and Spectroscopy
• Spectral dataset of young type Ib supernovae and their time evolution
• Astro-COLIBRI 2 -- an advanced platform for real-time multi-messenger discoveries
• The enigmatic double-peaked stripped-envelope SN 2023aew
• Massive stars exploding in a He-rich circumstellar medium. XI. Diverse evolution of five Ibn SNe 2020nxt, 2020taz, 2021bbv, 2023utc and 2024aej
• Application of Convolutional Neural Networks to time domain astrophysics. 2D image analysis of OGLE light curves
• Three Cases of Optical Periodic Modulation in Active Galactic Nuclei
• The Success of Optical Variability in Uncovering AGNs in Low-stellar Mass Galaxies
• A Newborn AGN in a Starforming Galaxy
• A search for gravitationally lensed supernovae within the Zwicky Transient Facility public survey
• AT2021acak: a candidate tidal disruption event found in the Zwicky Transient Facility survey
• A search for intermediate-mass black holes in compact stellar systems through optical emissions from tidal disruption events
• Early Identification of Optical Tidal Disruption Events: A science module for the Fink broker
• ELEPHANT: ExtragaLactic alErt Pipeline for Hostless AstroNomical Transients
• Hawai'i Supernova Flows: A Peculiar Velocity Survey Using Over a Thousand Supernovae in the Near-Infrared
• Exploring GRBs and supernovae connection: does a superluminous hypernova population exist?
• Multi-scale stamps for real-time classification of alert streams
• Real-Time Active Learning for optimised spectroscopic follow-up: Enhancing early SN Ia classification with the Fink broker
• Light Curve Classification with DistClassiPy: a new distance-based classifier
• The automation of optical transient discovery and classification in Rubin-era time-domain astronomy
• The DEHVILS in the Details: Type Ia Supernova Hubble Residual Comparisons and Mass Step Analysis in the Near-Infrared
• Discovery of the magnetic cataclysmic variable XMM J152737.4-205305.9 with a deep eclipse-like feature
• Searching for Supernovae in HETDEX Data Release 3
• Improving Astronomical Time-series Classification via Data Augmentation with Generative Adversarial Networks
• Observational diversity of bright long-lived Type II supernovae
• FALCO: a Foundation model of Astronomical Light Curves for time dOmain astronomy
• A Broker Integrated Algorithm for Gravitational Wave - Electromagnetic Counterpart Searches in O4a and O4b Runs
• Machine-directed gravitational-wave counterpart discovery
• SN 2022ngb: A faint, slowly evolving Type IIb supernova with a low-mass envelope
• Technosignature Searches with Real-time Alert Brokers
• SN 2023ixf: interaction signatures in the spectrum at 445 days
• RADAR-Radio Afterglow Detection and AI-driven Response: A Federated Framework for Gravitational Wave Event Follow-Up
• Multiband analysis of the O'Connell effect in 14 eclipsing binaries
• Hertzsprung gap stars in nearby galaxies and the Quest for Luminous Red Novae Progenitors
• Unlocking AGN Variability with Custom ZTF Photometry for High-Fidelity Light Curves and Robust Selection
• Machine Learning applications for Cataclysmic Variable discovery in the ZTF alert stream
• Multivariate time series transformer embeddings for light curves of periodic variable stars
• Contaminating Electromagnetic Transients in LISA Gravitational Wave Localization Volumes. I: The Intrinsic Rates
• AT 2021hdr: A candidate tidal disruption of a gas cloud by a binary super massive black hole system
• The Type IIn SN 2025cbj coincidence with the high-energy neutrino IceCube-250421A
• Illuminating the Mass Gap Through Deep Optical Constraint on a Neutron Star Merger Candidate S250206dm
• Large Language Model-driven Analysis of General Coordinates Network (GCN) Circulars
• Multiwavelength Characterization of a New Magnetic Cataclysmic Variable 2CXO J050740.7-091337
• The Solar System Notification Alert Processing System (SNAPS): Asteroid Population Outlier Detection
• VLA Observations Confirm AT 2023mfm as an Off-nuclear Tidal Disruption Event
• Electron-capture Supernova Candidates from Light Curves: Implications for Their Progenitors and Explosion Properties
• Semi-Supervised Learning for Lensed Quasar Detection
• The Large Array Survey Telescope-Pipeline. II. Image Subtraction and Transient Detection
• SN 2021tsz: A luminous, short photospheric phase Type II supernova in a low-metallicity host
• Types of Transients in the Centers of Post-Starburst and Quiescent Balmer-Strong Galaxies
• Subluminous Type IIP SN 2024abfl as a Result of a Significantly Low-energy Fe-core Collapse
• Finding Fast Transients in Real Time Using Novel Light Curve Analysis Algorithm