Astroconformer: The Prospects of Analyzing Stellar Light Curves with Transformer-Based Deep Learning Models

Abstract

Stellar light curves contain valuable information about oscillations and granulation, offering insights into stars' internal structures and evolutionary states. Traditional asteroseismic techniques, primarily focused on power spectral analysis, often overlook the crucial phase information in these light curves. Addressing this gap, recent machine learning applications, particularly those using Convolutional Neural Networks (CNNs), have made strides in inferring stellar properties from light curves. However, CNNs are limited by their localized feature extraction capabilities. In response, we introduce $\text{Astroconformer}$, a Transformer-based deep learning framework, specifically designed to capture long-range dependencies in stellar light curves. Our empirical analysis centers on estimating surface gravity ($\log g$), using a dataset derived from single-quarter Kepler light curves with $\log g$ values ranging from 0.2 to 4.4. $\text{Astroconformer}$ demonstrates superior performance, achieving a root-mean-square-error (RMSE) of 0.017 dex at $\log g\approx 3$ in data-rich regimes and up to 0.1 dex in sparser areas. This performance surpasses both K-nearest neighbor models and advanced CNNs. Ablation studies highlight the influence of receptive field size on model effectiveness, with larger fields correlating to improved results. $\text{Astroconformer}$ also excels in extracting ${\nu }_{\max }$ with high precision. It achieves less than 2% relative median absolute error for 90-day red giant light curves. Notably, the error remains under 3% for 30-day light curves, whose oscillations are undetectable by a conventional pipeline in 30% cases. Furthermore, the attention mechanisms in $\text{Astroconformer}$ align closely with the characteristics of stellar oscillations and granulation observed in light curves.

15 pages, 10 figures, Accepted by MNRAS

References in corpus (15)

• Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift
• The Zwicky Transient Facility: System Overview, Performance, and First Results
• The K2 Mission: Characterization and Early results
• A Time Series is Worth 64 Words: Long-term Forecasting with Transformers
• Conformer: Convolution-augmented Transformer for Speech Recognition
• The connection between stellar granulation and oscillation as seen by the Kepler mission
• RoFormer: Enhanced Transformer with Rotary Position Embedding
• Asteroseismology of solar-type stars
• Weakened magnetic braking supported by asteroseismic rotation rates of Kepler dwarfs
• 30 to 100-kG magnetic fields in the cores of red giant stars
• DeepNet: Scaling Transformers to 1,000 Layers
• Asteroseismology of luminous red giants with Kepler I: Long Period Variables with radial and non-radial modes
• A Search for Red Giant Solar-like Oscillations in All Kepler Data
• On the Generalization Mystery in Deep Learning
• The Swan: Data-Driven Inference of Stellar Surface Gravities for Cool Stars from Photometric Light Curves

Cited by in corpus (7)

• Identifying Light-curve Signals with a Deep Learning Based Object Detection Algorithm. II. A General Light Curve Classification Framework
• FALCO: a Foundation model of Astronomical Light Curves for time dOmain astronomy
• Uncertainty estimation for time series classification: Exploring predictive uncertainty in transformer-based models for variable stars
• ABC-SN: Attention Based Classifier for Supernova Spectra
• $\text{Exoformer}$: Accelerating Bayesian atmospheric retrievals with transformer neural networks
• Exploring Late Stellar Evolution in the Era of Large Surveys: Machine Learning Prospects for Hot Subdwarfs and White Dwarfs
• Sequence modeling of higher-order wave modes of binary black hole mergers