Multivariate time series transformer embeddings for light curves of periodic variable stars

Abstract

Astronomical surveys produce time series data by observing stellar objects across multiple photometry bands. Foundational transformer-based models, such as Astromer, encode each time series as a sequence of embeddings to classify sources. However, such models operate independently on each band and therefore do not use information across wavelengths or filters. We extend the single-band Astromer framework by introducing a fusion layer that combines single photometry band observations into a unified sequence representation, thus enabling multiband analysis for downstream tasks, namely periodic variable star classification. The challenge in adapting the encoder for multiband data lies in coordinating information across bands observed at asynchronous times. We pre-trained various multiband neural network models on 600 000 high signal-to-noise light curves from the Massive Compact Halo Object (MACHO) survey and fine-tuned them using labeled data from the Alcock catalog (derived from MACHO) and from the Asteroid Terrestrial-impact Last Alert System survey. Our results show that both proposed multiband architectures outperform the single-band models by approximately ten percentage points in the F1-score. Jointly pre-trained multiband encoders further improve the performance compared to a collection of independently pre-trained single-band encoders, reaching an improvement of 23 percentage points in the F1-score. These results demonstrate a trade-off in the training speed and classification accuracy between single-band and multiband encoders, with multiband models improving the F1-score performance by ten percentage points at the cost of increased pretraining time. Our results show the great potential of transformer-based multiband neural network architectures for classification tasks regarding future large-scale time-domain surveys with a greater variety of variable stars.

References in corpus (26)

• The Gaia mission
• LSST: from Science Drivers to Reference Design and Anticipated Data Products
• The Zwicky Transient Facility: System Overview, Performance, and First Results
• ATLAS: A High-Cadence All-Sky Survey System
• Possible Gravitational Microlensing of a Star in the Large Magellanic Cloud
• A First Catalog of Variable Stars Measured by the Asteroid Terrestrial-impact Last Alert System (ATLAS)
• On Machine-Learned Classification of Variable Stars with Sparse and Noisy Time-Series Data
• The Automatic Learning for the Rapid Classification of Events (ALeRCE) Alert Broker
• SuperNNova: an open-source framework for Bayesian, Neural Network based supernova classification
• Optimization of the Observing Cadence for the Rubin Observatory Legacy Survey of Space and Time: a pioneering process of community-focused experimental design
• Alert Classification for the ALeRCE Broker System: The Light Curve Classifier
• A recurrent neural network for classification of unevenly sampled variable stars
• Avocado: Photometric Classification of Astronomical Transients with Gaussian Process Augmentation
• The MACHO Project Large Magellanic Cloud variable star inventory. XI. Frequency analysis of the fundamental mode RR Lyrae stars
• Deep Recurrent Neural Networks for Supernovae Classification
• SuperRAENN: A Semi-supervised Supernova Photometric Classification Pipeline Trained on Pan-STARRS1 Medium Deep Survey Supernovae
• The EPOCH Project: I. Periodic variable stars in the EROS-2 LMC database
• A novel variability-based method for quasar selection: evidence for a rest frame ~54 day characteristic timescale
• Scalable End-to-end Recurrent Neural Network for Variable star classification
• On Neural Architectures for Astronomical Time-series Classification with Application to Variable Stars
• ASTROMER: A transformer-based embedding for the representation of light curves
• The QUEST-La Silla AGN Variability Survey: selection of AGN candidates through optical variability
• Deep Attention-Based Supernovae Classification of Multi-Band Light-Curves
• ATAT: Astronomical Transformer for time series And Tabular data
• The effect of phased recurrent units in the classification of multiple catalogs of astronomical lightcurves
• Uncertainty estimation for time series classification: Exploring predictive uncertainty in transformer-based models for variable stars