ATAT: Astronomical Transformer for time series And Tabular data

Abstract

The advent of next-generation survey instruments, such as the Vera C. Rubin Observatory and its Legacy Survey of Space and Time (LSST), is opening a window for new research in time-domain astronomy. The Extended LSST Astronomical Time-Series Classification Challenge (ELAsTiCC) was created to test the capacity of brokers to deal with a simulated LSST stream. We describe ATAT, the Astronomical Transformer for time series And Tabular data, a classification model conceived by the ALeRCE alert broker to classify light-curves from next-generation alert streams. ATAT was tested in production during the first round of the ELAsTiCC campaigns. ATAT consists of two Transformer models that encode light curves and features using novel time modulation and quantile feature tokenizer mechanisms, respectively. ATAT was trained on different combinations of light curves, metadata, and features calculated over the light curves. We compare ATAT against the current ALeRCE classifier, a Balanced Hierarchical Random Forest (BHRF) trained on human-engineered features derived from light curves and metadata. When trained on light curves and metadata, ATAT achieves a macro F1-score of 82.9 +- 0.4 in 20 classes, outperforming the BHRF model trained on 429 features, which achieves a macro F1-score of 79.4 +- 0.1. The use of Transformer multimodal architectures, combining light curves and tabular data, opens new possibilities for classifying alerts from a new generation of large etendue telescopes, such as the Vera C. Rubin Observatory, in real-world brokering scenarios.

References in corpus (33)

• Five-Year Wilkinson Microwave Anisotropy Probe (WMAP) Observations: Cosmological Interpretation
• The Astropy Project: Sustaining and Growing a Community-oriented Open-source Project and the Latest Major Release (v5.0) of the Core Package
• LSST: from Science Drivers to Reference Design and Anticipated Data Products
• The Zwicky Transient Facility: System Overview, Performance, and First Results
• The Zwicky Transient Facility: Science Objectives
• Rotation-invariant convolutional neural networks for galaxy morphology prediction
• The Automatic Learning for the Rapid Classification of Events (ALeRCE) Alert Broker
• SuperNNova: an open-source framework for Bayesian, Neural Network based supernova classification
• The delay of shock breakout due to circumstellar material seen in most Type II Supernovae
• RAPID: Early Classification of Explosive Transients using Deep Learning
• Alert Classification for the ALeRCE Broker System: The Light Curve Classifier
• A recurrent neural network for classification of unevenly sampled variable stars
• Deep-HiTS: Rotation Invariant Convolutional Neural Network for Transient Detection
• Avocado: Photometric Classification of Astronomical Transients with Gaussian Process Augmentation
• Fink, a new generation of broker for the LSST community
• Deep Recurrent Neural Networks for Supernovae Classification
• Transient processing and analysis using $\text{AMPEL}$: Alert Management, Photometry and Evaluation of Lightcurves
• Machine Learning-based Brokers for Real-time Classification of the LSST Alert Stream
• Computational Intelligence Challenges and Applications on Large-Scale Astronomical Time Series Databases
• The ANTARES Astronomical Time-Domain Event Broker
• Scalable End-to-end Recurrent Neural Network for Variable star classification
• Searching for changing-state AGNs in massive datasets -- I: applying deep learning and anomaly detection techniques to find AGNs with anomalous variability behaviours
• Deep Learning for Image Sequence Classification of Astronomical Events
• Active Anomaly Detection for time-domain discoveries
• On Neural Architectures for Astronomical Time-series Classification with Application to Variable Stars
• ASTROMER: A transformer-based embedding for the representation of light curves
• Clustering Based Feature Learning on Variable Stars
• Deep Attention-Based Supernovae Classification of Multi-Band Light-Curves
• Classifying Image Sequences of Astronomical Transients with Deep Neural Networks
• Persistent and occasional: searching for the variable population of the ZTF/4MOST sky using ZTF data release 11
• The effect of phased recurrent units in the classification of multiple catalogs of astronomical lightcurves
• MANTRA: A Machine Learning reference lightcurve dataset for astronomical transient event recognition
• DELIGHT: Deep Learning Identification of Galaxy Hosts of Transients using Multi-resolution Images

Cited by in corpus (10)

• Transient Classifiers for Fink: Benchmarks for LSST
• ORACLE: A Real-Time, Hierarchical, Deep-Learning Photometric Classifier for the LSST
• Transfer Learning for Transient Classification: From Simulations to Real Data and ZTF to LSST
• Testing and Combining Transient Spectral Classification Tools on 4MOST-like Blended Spectra
• Real-Time Active Learning for optimised spectroscopic follow-up: Enhancing early SN Ia classification with the Fink broker
• Deep learning-based astronomical multimodal data fusion: A comprehensive review
• Leveraging pre-trained vision Transformers for multi-band photometric light curve classification
• Uncertainty estimation for time series classification: Exploring predictive uncertainty in transformer-based models for variable stars
• Multivariate time series transformer embeddings for light curves of periodic variable stars
• Exploring Late Stellar Evolution in the Era of Large Surveys: Machine Learning Prospects for Hot Subdwarfs and White Dwarfs