Uncertainty estimation for time series classification: Exploring predictive uncertainty in transformer-based models for variable stars

Abstract

Classifying variable stars is key for understanding stellar evolution and galactic dynamics. With the demands of large astronomical surveys, machine learning models, especially attention-based neural networks, have become the state-of-the-art. While achieving high accuracy is crucial, enhancing model interpretability and uncertainty estimation is equally important to ensure that insights are both reliable and comprehensible. We aim to enhance transformer-based models for classifying astronomical light curves by incorporating uncertainty estimation techniques to detect misclassified instances. We tested our methods on labeled datasets from MACHO, OGLE-III, and ATLAS, introducing a framework that significantly improves the reliability of automated classification for the next-generation surveys. We used Astromer, a transformer-based encoder designed for capturing representations of single-band light curves. We enhanced its capabilities by applying three methods for quantifying uncertainty: Monte Carlo Dropout (MC Dropout), Hierarchical Stochastic Attention (HSA), and a novel hybrid method combining both approaches, which we have named Hierarchical Attention with Monte Carlo Dropout (HA-MC Dropout). We compared these methods against a baseline of deep ensembles (DEs). To estimate uncertainty estimation scores for the misclassification task, we selected Sampled Maximum Probability (SMP), Probability Variance (PV), and Bayesian Active Learning by Disagreement (BALD) as uncertainty estimates. In predictive performance tests, HA-MC Dropout outperforms the baseline, achieving macro F1-scores of 79.8+-0.5 on OGLE, 84+-1.3 on ATLAS, and 76.6+-1.8 on MACHO. When comparing the PV score values, the quality of uncertainty estimation by HA-MC Dropout surpasses that of all other methods, with improvements of 2.5+-2.3 for MACHO, 3.3+-2.1 for ATLAS and 8.5+-1.6 for OGLE-III.

References in corpus (24)

• LSST: from Science Drivers to Reference Design and Anticipated Data Products
• Ensemble deep learning: A review
• The MACHO Project: Microlensing Results from 5.7 Years of LMC Observations
• A First Catalog of Variable Stars Measured by the Asteroid Terrestrial-impact Last Alert System (ATLAS)
• SuperNNova: an open-source framework for Bayesian, Neural Network based supernova classification
• A recurrent neural network for classification of unevenly sampled variable stars
• Optimizing spectroscopic follow-up strategies for supernova photometric classification with active learning
• Cepheid Variables in the Flared Outer Disk of our Galaxy
• Astronomia ex machina: a history, primer, and outlook on neural networks in astronomy
• Unveiling the Distinct Formation Pathways of the Inner and Outer Discs of the Milky Way with Bayesian Machine Learning
• Active Learning to Overcome Sample Selection Bias: Application to Photometric Variable Star Classification
• A simple and robust method for automated photometric classification of supernovae using neural networks
• AstroCLIP: A Cross-Modal Foundation Model for Galaxies
• Scalable End-to-end Recurrent Neural Network for Variable star classification
• Automated Classification of Periodic Variable Stars detected by the Wide-field Infrared Survey Explorer
• Deep Learning for Image Sequence Classification of Astronomical Events
• Transient-optimised real-bogus classification with Bayesian Convolutional Neural Networks -- sifting the GOTO candidate stream
• ASTROMER: A transformer-based embedding for the representation of light curves
• Fink: early supernovae Ia classification using active learning
• Deep Attention-Based Supernovae Classification of Multi-Band Light-Curves
• The High Cadence Transient Survey (HITS): Compilation and characterization of light-curve catalogs
• ATAT: Astronomical Transformer for time series And Tabular data
• Astroconformer: The Prospects of Analyzing Stellar Light Curves with Transformer-Based Deep Learning Models
• The effect of phased recurrent units in the classification of multiple catalogs of astronomical lightcurves

Cited by in corpus (1)

• Multivariate time series transformer embeddings for light curves of periodic variable stars