Informative regularization for a multi-layer perceptron RR Lyrae classifier under data shift

Abstract

In recent decades, machine learning has provided valuable models and algorithms for processing and extracting knowledge from time-series surveys. Different classifiers have been proposed and performed to an excellent standard. Nevertheless, few papers have tackled the data shift problem in labeled training sets, which occurs when there is a mismatch between the data distribution in the training set and the testing set. This drawback can damage the prediction performance in unseen data. Consequently, we propose a scalable and easily adaptable approach based on an informative regularization and an ad-hoc training procedure to mitigate the shift problem during the training of a multi-layer perceptron for RR Lyrae classification. We collect ranges for characteristic features to construct a symbolic representation of prior knowledge, which was used to model the informative regularizer component. Simultaneously, we design a two-step back-propagation algorithm to integrate this knowledge into the neural network, whereby one step is applied in each epoch to minimize classification error, while another is applied to ensure regularization. Our algorithm defines a subset of parameters (a mask) for each loss function. This approach handles the forgetting effect, which stems from a trade-off between these loss functions (learning from data versus learning expert knowledge) during training. Experiments were conducted using recently proposed shifted benchmark sets for RR Lyrae stars, outperforming baseline models by up to 3\% through a more reliable classifier. Our method provides a new path to incorporate knowledge from characteristic features into artificial neural networks to manage the underlying data shift problem.

References in corpus (38)

• Scikit-learn: Machine Learning in Python
• The Wide-field Infrared Survey Explorer (WISE): Mission Description and Initial On-orbit Performance
• K-corrections and filter transformations in the ultraviolet, optical, and near infrared
• VISTA Variables in the Via Lactea (VVV): The public ESO near-IR variability survey of the Milky Way
• The MACHO Project LMC Microlensing Results from the First Two Years and the Nature of the Galactic Dark Halo
• On Machine-Learned Classification of Variable Stars with Sparse and Noisy Time-Series Data
• Automated supervised classification of variable stars I. Methodology
• Machine-Learned Identification of RR Lyrae Stars from Sparse, Multi-band Data: the PS1 Sample
• Automating Discovery and Classification of Transients and Variable Stars in the Synoptic Survey Era
• In Search of the Real Inductive Bias: On the Role of Implicit Regularization in Deep Learning
• A recurrent neural network for classification of unevenly sampled variable stars
• The Carnegie-Chicago Hubble Program. I. An Independent Approach to the Extragalactic Distance Scale Using only Population II Distance Indicators
• Machine Learning-based Brokers for Real-time Classification of the LSST Alert Stream
• A Package for the Automated Classification of Periodic Variable Stars
• Active Learning to Overcome Sample Selection Bias: Application to Photometric Variable Star Classification
• Mapping the outer bulge with RRab stars from the VVV Survey
• Scalable End-to-end Recurrent Neural Network for Variable star classification
• Supervised detection of anomalous light-curves in massive astronomical catalogs
• Deep multi-survey classification of variable stars
• Deep Learning for Image Sequence Classification of Astronomical Events
• Automated Classification of Periodic Variable Stars detected by the Wide-field Infrared Survey Explorer
• A machine learned classifier for RR Lyrae in the VVV survey
• On Neural Architectures for Astronomical Time-series Classification with Application to Variable Stars
• Automated supervised classification of variable stars II. Application to the OGLE database
• An improved quasar detection method in EROS-2 and MACHO LMC datasets
• Clustering Based Feature Learning on Variable Stars
• The Optical Gravitational Lensing Experiment. Final Reductions of the OGLE-III Data
• Meta Classification for Variable Stars
• Improving Deep Learning Models via Constraint-Based Domain Knowledge: a Brief Survey
• Data Representativity for Machine Learning and AI Systems
• Real-Time Value-Driven Data Augmentation in the Era of LSST
• Automatic Survey-Invariant Variable Star Classification
• Classification of Periodic Variable Stars with Novel Cyclic-Permutation Invariant Neural Networks
• Uncertain classification of Variable Stars: handling observational GAPS and noise
• Automatic classification of eclipsing binary stars using deep learning methods
• Near-Infrared Search for Fundamental-mode RR Lyrae Stars Toward the Inner Bulge by Deep Learning
• Constraint-Based Regularization of Neural Networks
• Informative Bayesian model selection for RR Lyrae star classifiers