Informative regularization for a multi-layer perceptron RR Lyrae classifier under data shift

Abstract

In recent decades, machine learning has provided valuable models and algorithms for processing and extracting knowledge from time-series surveys. Different classifiers have been proposed and performed to an excellent standard. Nevertheless, few papers have tackled the data shift problem in labeled training sets, which occurs when there is a mismatch between the data distribution in the training set and the testing set. This drawback can damage the prediction performance in unseen data. Consequently, we propose a scalable and easily adaptable approach based on an informative regularization and an ad-hoc training procedure to mitigate the shift problem during the training of a multi-layer perceptron for RR Lyrae classification. We collect ranges for characteristic features to construct a symbolic representation of prior knowledge, which was used to model the informative regularizer component. Simultaneously, we design a two-step back-propagation algorithm to integrate this knowledge into the neural network, whereby one step is applied in each epoch to minimize classification error, while another is applied to ensure regularization. Our algorithm defines a subset of parameters (a mask) for each loss function. This approach handles the forgetting effect, which stems from a trade-off between these loss functions (learning from data versus learning expert knowledge) during training. Experiments were conducted using recently proposed shifted benchmark sets for RR Lyrae stars, outperforming baseline models by up to 3\% through a more reliable classifier. Our method provides a new path to incorporate knowledge from characteristic features into artificial neural networks to manage the underlying data shift problem.

References in corpus (16)

• K-corrections and filter transformations in the ultraviolet, optical, and near infrared
• Automated supervised classification of variable stars I. Methodology
• In Search of the Real Inductive Bias: On the Role of Implicit Regularization in Deep Learning
• A recurrent neural network for classification of unevenly sampled variable stars
• Scalable End-to-end Recurrent Neural Network for Variable star classification
• Supervised detection of anomalous light-curves in massive astronomical catalogs
• A machine learned classifier for RR Lyrae in the VVV survey
• Automated supervised classification of variable stars II. Application to the OGLE database
• An improved quasar detection method in EROS-2 and MACHO LMC datasets
• The Optical Gravitational Lensing Experiment. Final Reductions of the OGLE-III Data
• Improving Deep Learning Models via Constraint-Based Domain Knowledge: a Brief Survey
• Automatic Survey-Invariant Variable Star Classification
• Uncertain classification of Variable Stars: handling observational GAPS and noise
• Automatic classification of eclipsing binary stars using deep learning methods
• Near-Infrared Search for Fundamental-mode RR Lyrae Stars Toward the Inner Bulge by Deep Learning
• Informative Bayesian model selection for RR Lyrae star classifiers