Learn from every mistake! Hierarchical information combination in astronomy
arXiv:1702.04480 · doi:10.1017/S1743921317000242
Abstract
Throughout the processing and analysis of survey data, a ubiquitous issue nowadays is that we are spoilt for choice when we need to select a methodology for some of its steps. The alternative methods usually fail and excel in different data regions, and have various advantages and drawbacks, so a combination that unites the strengths of all while suppressing the weaknesses is desirable. We propose to use a two-level hierarchy of learners. Its first level consists of training and applying the possible base methods on the first part of a known set. At the second level, we feed the output probability distributions from all base methods to a second learner trained on the remaining known objects. Using classification of variable stars and photometric redshift estimation as examples, we show that the hierarchical combination is capable of achieving general improvement over averaging-type combination methods, correcting systematics present in all base methods, is easy to train and apply, and thus, it is a promising tool in the astronomical "Big Data" era.
6 pages, 3 figures. To appear in the conference proceedings of the IAU Symposium 325 AstroInformatics (2016 October 20-24, Sorrento, Italy)
References in corpus (8)
- EAZY: A Fast, Public Photometric Redshift Code
- Automated Transient Identification in the Dark Energy Survey
- SOMz: photometric redshift PDFs with self organizing maps and random atlas
- Exhausting the Information: Novel Bayesian Combination of Photometric Redshift PDFs
- Automated classification of Hipparcos unsolved variables
- Detection of Dispersed Radio Pulses: A machine learning approach to candidate identification and classification
- ASTErIsM - Application of topometric clustering algorithms in automatic galaxy detection and classification
- Sparse Representation of Photometric Redshift PDFs: Preparing for Petascale Astronomy