On Neural Architectures for Astronomical Time-series Classification with Application to Variable Stars

Abstract

Despite the utility of neural networks (NNs) for astronomical time-series classification, the proliferation of learning architectures applied to diverse datasets has thus far hampered a direct intercomparison of different approaches. Here we perform the first comprehensive study of variants of NN-based learning and inference for astronomical time-series, aiming to provide the community with an overview on relative performance and, hopefully, a set of best-in-class choices for practical implementations. In both supervised and self-supervised contexts, we study the effects of different time-series-compatible layer choices, namely the dilated temporal convolutional neural network (dTCNs), Long-Short Term Memory (LSTM) NNs, Gated Recurrent Units (GRUs) and temporal convolutional NNs (tCNNs). We also study the efficacy and performance of encoder-decoder (i.e., autoencoder) networks compared to direct classification networks, different pathways to include auxiliary (non-time-series) metadata, and different approaches to incorporate multi-passband data (i.e., multiple time-series per source). Performance---applied to a sample of 17,604 variable stars from the MACHO survey across 10 imbalanced classes---is measured in training convergence time, classification accuracy, reconstruction error, and generated latent variables. We find that networks with Recurrent NN (RNNs) generally outperform dTCNs and, in many scenarios, yield to similar accuracy as tCNNs. In learning time and memory requirements, convolution-based layers are more performant. We conclude by discussing the advantages and limitations of deep architectures for variable star classification, with a particular eye towards next-generation surveys such as LSST, WFIRST and ZTF2.

arXiv commentSubmitted to ApJS

References in corpus (67)

• Scikit-learn: Machine Learning in Python
• Deep Learning in Neural Networks: An Overview
• The NumPy array: a structure for efficient numerical computation
• LSST: from Science Drivers to Reference Design and Anticipated Data Products
• Final Results from the Hubble Space Telescope Key Project to Measure the Hubble Constant
• Large Magellanic Cloud Cepheid Standards Provide a 1% Foundation for the Determination of the Hubble Constant and Stronger Evidence for Physics Beyond LambdaCDM
• Accurate masses and radii of normal stars: modern results and applications
• Fast and scalable Gaussian process modeling with applications to astronomical time series
• Rotation-invariant convolutional neural networks for galaxy morphology prediction
• Milky Way Cepheid Standards for Measuring Cosmic Distances and Application to Gaia DR2: Implications for the Hubble Constant
• Bayesian Blocks, A New Method to Analyze Structure in Photon Counting Data
• Improving galaxy morphologies for SDSS with Deep Learning
• Deep Learning for Real-time Gravitational Wave Detection and Parameter Estimation: Results with Advanced LIGO Data
• Identifying Exoplanets with Deep Learning: A Five Planet Resonant Chain around Kepler-80 and an Eighth Planet around Kepler-90
• Distances, ages, and epoch of formation of globular clusters
• On Machine-Learned Classification of Variable Stars with Sparse and Noisy Time-Series Data
• Deep Neural Networks to Enable Real-time Multimessenger Astrophysics
• Periodograms for Multiband Astronomical Time Series
• Matching matched filtering with deep networks in gravitational-wave astronomy
• Star-galaxy Classification Using Deep Convolutional Neural Networks
• Automated supervised classification of variable stars I. Methodology
• CMU DeepLens: Deep Learning For Automatic Image-based Galaxy-Galaxy Strong Lens Finding
• A three-dimensional map of the Milky Way using classical Cepheid variable stars
• Photometric redshifts from SDSS images using a Convolutional Neural Network
• SuperNNova: an open-source framework for Bayesian, Neural Network based supernova classification
• Random forest automated supervised classification of Hipparcos periodic variable stars
• Photometric redshift estimation via deep learning
• The MACHO Project First Year LMC Results: The Microlensing Rate and the Nature of the Galactic Dark Halo
• Convolutional neural networks: a magic bullet for gravitational-wave detection?
• RAPID: Early Classification of Explosive Transients using Deep Learning
• Classifying Radio Galaxies with Convolutional Neural Network
• A Review of the Distance and Structure of the Large Magellanic Cloud
• A recurrent neural network for classification of unevenly sampled variable stars
• Avocado: Photometric Classification of Astronomical Transients with Gaussian Process Augmentation
• CosmoGAN: creating high-fidelity weak lensing convergence maps using Generative Adversarial Networks
• K2 Variable Catalogue II: Machine Learning Classification of Variable Stars and Eclipsing Binaries in K2 Fields 0-4
• Deep Recurrent Neural Networks for Supernovae Classification
• Deep learning from 21-cm tomography of the Cosmic Dawn and Reionization
• Characteristics of the Galaxy according to Cepheids
• Finding high-redshift strong lenses in DES using convolutional neural networks
• Construction of a Calibrated Probabilistic Classification Catalog: Application to 50k Variable Sources in the All-Sky Automated Survey
• A Package for the Automated Classification of Periodic Variable Stars
• Applying Deep Learning to Fast Radio Burst Classification
• Deep Learning of Quasar Spectra to Discover and Characterize Damped Lya Systems
• A Deep Learning-based Reconstruction of Cosmic Ray-induced Air Showers
• Deep Learning Classification in Asteroseismology
• Variable stars across the observational HR diagram
• Scientific Domain Knowledge Improves Exoplanet Transit Classification with Deep Learning
• Identifying galaxies, quasars, and stars with machine learning: A new catalogue of classifications for 111 million SDSS sources without spectra
• A Near-Infrared Period-Luminosity Relation for Miras in NGC 4258, an Anchor for a New Distance Ladder
• Machine-learning Approaches to Exoplanet Transit Detection and Candidate Validation in Wide-field Ground-based Surveys
• Applying deep neural networks to the detection and space parameter estimation of compact binary coalescence with a network of gravitational wave detectors
• DASH: Deep Learning for the Automated Spectral Classification of Supernovae and their Hosts
• An analysis of feature relevance in the classification of astronomical transients with machine learning methods
• PELICAN: deeP architecturE for the LIght Curve ANalysis
• Automated Classification of Periodic Variable Stars detected by the Wide-field Infrared Survey Explorer
• Automated classification of Hipparcos unsolved variables
• A Hybrid Deep Learning Approach to Cosmological Constraints From Galaxy Redshift Surveys
• Anomaly Detection in the Open Supernova Catalog
• Denoising Gravitational Waves with Enhanced Deep Recurrent Denoising Auto-Encoders
• Eclipsing binaries in the MACHO database: New periods and classifications for 3031 systems in the Large Magellanic Cloud
• Denoising Gravitational Waves using Deep Learning with Recurrent Denoising Autoencoders
• Automated classification of periodic variable stars{Improved methodology for the automated classification of periodic variable stars}
• Into the Darkness: Classical and Type II Cepheids in the Zona Galactica Incognita
• Deep Neural Network Classifier for Variable Stars with Novelty Detection Capability
• Variable Star Classification Using Multi-View Metric Learning
• An Algorithm for the Visualization of Relevant Patterns in Astronomical Light Curves

Cited by in corpus (28)

• Fink, a new generation of broker for the LSST community
• Double-, triple-line spectroscopic candidates in the LAMOST Medium-Resolution Spectroscopic Survey
• ASTROMER: A transformer-based embedding for the representation of light curves
• The ZTF Source Classification Project: I. Methods and Infrastructure
• Classifying Kepler light curves for 12,000 A and F stars using supervised feature-based machine learning
• TESS Data for Asteroseismology (T'DA) Stellar Variability Classification Pipeline: Set-Up and Application to the Kepler Q9 Data
• Deep Attention-Based Supernovae Classification of Multi-Band Light-Curves
• Real-Time Detection of Anomalies in Large-Scale Transient Surveys
• A method for finding anomalous astronomical light curves and their analogs
• LSST Cadence Strategy Evaluations for AGN Time-series Data in Wide-Fast-Deep Field
• Classification of Periodic Variable Stars with Novel Cyclic-Permutation Invariant Neural Networks
• ATAT: Astronomical Transformer for time series And Tabular data
• The effect of phased recurrent units in the classification of multiple catalogs of astronomical lightcurves
• Identifying Light-curve Signals with a Deep Learning Based Object Detection Algorithm. II. A General Light Curve Classification Framework
• Searching for Anomalies in the ZTF Catalog of Periodic Variable Stars
• A probabilistic method for detecting solar-like oscillations using meaningful prior information
• Using a neural network approach to accelerate disequilibrium chemistry calculations in exoplanet atmospheres
• Leveraging Deep Learning for Time Series Extrinsic Regression in predicting photometric metallicity of Fundamental-mode RR Lyrae Stars
• Modeling the Time Evolution of Compact Binary Systems with Machine Learning
• Learning novel representations of variable sources from multi-modal $\text{Gaia}$ data via autoencoders
• Leveraging pre-trained vision Transformers for multi-band photometric light curve classification
• Real-time Light Curve Classification Framework for the Wide Field Survey Telescope Using Modified Semi-supervised Variational Auto-Encoder
• Heartbeat Stars Recognition Based on Recurrent Neural Networks: Method and Validation
• Multivariate time series transformer embeddings for light curves of periodic variable stars
• Informative regularization for a multi-layer perceptron RR Lyrae classifier under data shift
• Pushing the Limit of Asteroseismic Detection for Cool Dwarfs using TESS and Deep Learning
• Time-domain deep learning filtering of structured atmospheric noise for ground-based millimeter astronomy
• Finding Fast Transients in Real Time Using Novel Light Curve Analysis Algorithm