Informative Bayesian model selection for RR Lyrae star classifiers
arXiv:2105.11531 · doi:10.1093/mnras/stab320
Abstract
Machine learning has achieved an important role in the automatic classification of variable stars, and several classifiers have been proposed over the last decade. These classifiers have achieved impressive performance in several astronomical catalogues. However, some scientific articles have also shown that the training data therein contain multiple sources of bias. Hence, the performance of those classifiers on objects not belonging to the training data is uncertain, potentially resulting in the selection of incorrect models. Besides, it gives rise to the deployment of misleading classifiers. An example of the latter is the creation of open-source labelled catalogues with biased predictions. In this paper, we develop a method based on an informative marginal likelihood to evaluate variable star classifiers. We collect deterministic rules that are based on physical descriptors of RR Lyrae stars, and then, to mitigate the biases, we introduce those rules into the marginal likelihood estimation. We perform experiments with a set of Bayesian Logistic Regressions, which are trained to classify RR Lyraes, and we found that our method outperforms traditional non-informative cross-validation strategies, even when penalized models are assessed. Our methodology provides a more rigorous alternative to assess machine learning models using astronomical knowledge. From this approach, applications to other classes of variable stars and algorithmic improvements can be developed.
References in corpus (38)
- The Wide-field Infrared Survey Explorer (WISE): Mission Description and Initial On-orbit Performance
- Variational Inference: A Review for Statisticians
- A survey of cross-validation procedures for model selection
- A weakly informative default prior distribution for logistic and other regression models
- Bayes in the sky: Bayesian inference and model selection in cosmology
- SparsePak: A Formatted Fiber Field Unit for The WIYN Telescope Bench Spectrograph. I. Design, Construction, and Calibration
- A Widely Applicable Bayesian Information Criterion
- The MACHO Project LMC Microlensing Results from the First Two Years and the Nature of the Galactic Dark Halo
- The prior can generally only be understood in the context of the likelihood
- Data analysis recipes: Using Markov Chain Monte Carlo
- On Machine-Learned Classification of Variable Stars with Sparse and Noisy Time-Series Data
- Markov Chain Monte Carlo Methods for Bayesian Data Analysis in Astronomy
- Automated supervised classification of variable stars I. Methodology
- II: Bayesian Methods for Cosmological Parameter Estimation from Cosmic Microwave Background Measurements
- Automating Discovery and Classification of Transients and Variable Stars in the Synoptic Survey Era
- Isochrone ages for ~3 million stars with the second Gaia data release
- A recurrent neural network for classification of unevenly sampled variable stars
- Exploring the Variable Sky with LINEAR. II. Halo Structure and Substructure Traced by RR Lyrae Stars to 30 kpc
- Machine Learning-based Brokers for Real-time Classification of the LSST Alert Stream
- A Package for the Automated Classification of Periodic Variable Stars
- Active Learning to Overcome Sample Selection Bias: Application to Photometric Variable Star Classification
- Scalable End-to-end Recurrent Neural Network for Variable star classification
- Supervised detection of anomalous light-curves in massive astronomical catalogs
- Deep multi-survey classification of variable stars
- Automated Classification of Periodic Variable Stars detected by the Wide-field Infrared Survey Explorer
- Deep Learning for Image Sequence Classification of Astronomical Events
- A machine learned classifier for RR Lyrae in the VVV survey
- An improved quasar detection method in EROS-2 and MACHO LMC datasets
- Clustering Based Feature Learning on Variable Stars
- Unsupervised Classification of Variable Stars
- FATS: Feature Analysis for Time Series
- Meta Classification for Variable Stars
- Automatic Survey-Invariant Variable Star Classification
- A Bayesian Framework for Exoplanet Direct Detection and Non-Detection
- Uncertain classification of Variable Stars: handling observational GAPS and noise
- Classification of Multiwavelength Transients with Machine Learning
- Computational statistics using the Bayesian Inference Engine
- Faint Object Detection in Multi-Epoch Observations via Catalog Data Fusion