Astroconformer: The Prospects of Analyzing Stellar Light Curves with Transformer-Based Deep Learning Models

Abstract

Stellar light curves contain valuable information about oscillations and granulation, offering insights into stars' internal structures and evolutionary states. Traditional asteroseismic techniques, primarily focused on power spectral analysis, often overlook the crucial phase information in these light curves. Addressing this gap, recent machine learning applications, particularly those using Convolutional Neural Networks (CNNs), have made strides in inferring stellar properties from light curves. However, CNNs are limited by their localized feature extraction capabilities. In response, we introduce $\text{Astroconformer}$, a Transformer-based deep learning framework, specifically designed to capture long-range dependencies in stellar light curves. Our empirical analysis centers on estimating surface gravity ($\log g$), using a dataset derived from single-quarter Kepler light curves with $\log g$ values ranging from 0.2 to 4.4. $\text{Astroconformer}$ demonstrates superior performance, achieving a root-mean-square-error (RMSE) of 0.017 dex at $\log g\approx 3$ in data-rich regimes and up to 0.1 dex in sparser areas. This performance surpasses both K-nearest neighbor models and advanced CNNs. Ablation studies highlight the influence of receptive field size on model effectiveness, with larger fields correlating to improved results. $\text{Astroconformer}$ also excels in extracting ${\nu }_{\max }$ with high precision. It achieves less than 2% relative median absolute error for 90-day red giant light curves. Notably, the error remains under 3% for 30-day light curves, whose oscillations are undetectable by a conventional pipeline in 30% cases. Furthermore, the attention mechanisms in $\text{Astroconformer}$ align closely with the characteristics of stellar oscillations and granulation observed in light curves.

arXiv comment15 pages, 10 figures, Accepted by MNRAS

References in corpus (43)

• Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift
• An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale
• Decoupled Weight Decay Regularization
• A Simple Framework for Contrastive Learning of Visual Representations
• Deep Residual Learning for Image Recognition
• LSST: from Science Drivers to Reference Design and Anticipated Data Products
• Deep learning for time series classification: a review
• The Zwicky Transient Facility: System Overview, Performance, and First Results
• The K2 Mission: Characterization and Early results
• Kepler Mission Design, Realized Photometric Performance, and Early Science
• The CoRoT satellite in flight : description and performance
• Gravity modes as a way to distinguish between hydrogen- and helium-burning red giant stars
• A Time Series is Worth 64 Words: Long-term Forecasting with Transformers
• Asteroseismology of Solar-Type and Red-Giant Stars
• Conformer: Convolution-augmented Transformer for Speech Recognition
• Probing the interior physics of stars through asteroseismology
• Revised Stellar Properties of Kepler Targets for the Q1-17 (DR25) Transit Detection Run
• Preparation of Kepler lightcurves for asteroseismic analyses
• The connection between stellar granulation and oscillation as seen by the Kepler mission
• Asteroseismology of 16000 Kepler Red Giants: Global Oscillation Parameters, Masses, and Radii
• RoFormer: Enhanced Transformer with Rotary Position Embedding
• Theoretical amplitudes and lifetimes of non-radial solar-like oscillations in red giants
• Granulation in Red Giants: observations by the Kepler mission and 3D convection simulations
• Asteroseismology Can Reveal Strong Internal Magnetic Fields in Red Giant Stars
• Asteroseismology of solar-type stars
• Core rotation braking on the red giant branch for various mass ranges
• A prevalence of dynamo-generated magnetic fields in the cores of intermediate-mass stars
• Amplitudes of Stellar Oscillations: The Implications for Asteroseismology
• Characterisation of red-giant stars in the public Kepler data
• Weakened magnetic braking supported by asteroseismic rotation rates of Kepler dwarfs
• 30 to 100-kG magnetic fields in the cores of red giant stars
• Deep Learning Classification in Asteroseismology Using an Improved Neural Network: Results on 15000 Kepler Red Giants and Applications to K2 and TESS Data
• On the detection of Lorentzian profiles in a power spectrum: A Bayesian approach using ignorance priors
• Solar-like oscillations in red giants observed with \textit{Kepler}: influence of increased timespan on global oscillation parameters
• Machine Learning Techniques for Stellar Light Curve Classification
• Internal rotation of subdwarf B stars: limiting cases and asteroseismological consequences
• DeepNet: Scaling Transformers to 1,000 Layers
• Asteroseismology of luminous red giants with Kepler I: Long Period Variables with radial and non-radial modes
• A Search for Red Giant Solar-like Oscillations in All Kepler Data
• FliPer: A global measure of power density to estimate surface gravities of main-sequence Solar-like stars and red giants
• On the Generalization Mystery in Deep Learning
• Inference of stellar parameters from brightness variations
• The Swan: Data-Driven Inference of Stellar Surface Gravities for Cool Stars from Photometric Light Curves

Cited by in corpus (7)

• Identifying Light-curve Signals with a Deep Learning Based Object Detection Algorithm. II. A General Light Curve Classification Framework
• FALCO: a Foundation model of Astronomical Light Curves for time dOmain astronomy
• Uncertainty estimation for time series classification: Exploring predictive uncertainty in transformer-based models for variable stars
• ABC-SN: Attention Based Classifier for Supernova Spectra
• $\text{Exoformer}$: Accelerating Bayesian atmospheric retrievals with transformer neural networks
• Exploring Late Stellar Evolution in the Era of Large Surveys: Machine Learning Prospects for Hot Subdwarfs and White Dwarfs
• Sequence modeling of higher-order wave modes of binary black hole mergers