Multivariate time series transformer embeddings for light curves of periodic variable stars

Abstract

Astronomical surveys produce time series data by observing stellar objects across multiple photometry bands. Foundational transformer-based models, such as Astromer, encode each time series as a sequence of embeddings to classify sources. However, such models operate independently on each band and therefore do not use information across wavelengths or filters. We extend the single-band Astromer framework by introducing a fusion layer that combines single photometry band observations into a unified sequence representation, thus enabling multiband analysis for downstream tasks, namely periodic variable star classification. The challenge in adapting the encoder for multiband data lies in coordinating information across bands observed at asynchronous times. We pre-trained various multiband neural network models on 600 000 high signal-to-noise light curves from the Massive Compact Halo Object (MACHO) survey and fine-tuned them using labeled data from the Alcock catalog (derived from MACHO) and from the Asteroid Terrestrial-impact Last Alert System survey. Our results show that both proposed multiband architectures outperform the single-band models by approximately ten percentage points in the F1-score. Jointly pre-trained multiband encoders further improve the performance compared to a collection of independently pre-trained single-band encoders, reaching an improvement of 23 percentage points in the F1-score. These results demonstrate a trade-off in the training speed and classification accuracy between single-band and multiband encoders, with multiband models improving the F1-score performance by ten percentage points at the cost of increased pretraining time. Our results show the great potential of transformer-based multiband neural network architectures for classification tasks regarding future large-scale time-domain surveys with a greater variety of variable stars.

References in corpus (12)

• The Gaia mission
• The Zwicky Transient Facility: System Overview, Performance, and First Results
• Optimization of the Observing Cadence for the Rubin Observatory Legacy Survey of Space and Time: a pioneering process of community-focused experimental design
• A recurrent neural network for classification of unevenly sampled variable stars
• The EPOCH Project: I. Periodic variable stars in the EROS-2 LMC database
• Scalable End-to-end Recurrent Neural Network for Variable star classification
• ASTROMER: A transformer-based embedding for the representation of light curves
• The QUEST-La Silla AGN Variability Survey: selection of AGN candidates through optical variability
• Deep Attention-Based Supernovae Classification of Multi-Band Light-Curves
• ATAT: Astronomical Transformer for time series And Tabular data
• The effect of phased recurrent units in the classification of multiple catalogs of astronomical lightcurves
• Uncertainty estimation for time series classification: Exploring predictive uncertainty in transformer-based models for variable stars