A Study on Classification in Imbalanced and Partially-Labelled Data Streams
arXiv:1307.8012 · doi:10.1109/SMC.2013.260
Abstract
The domain of radio astronomy is currently facing significant computational challenges, foremost amongst which are those posed by the development of the world's largest radio telescope, the Square Kilometre Array (SKA). Preliminary specifications for this instrument suggest that the final design will incorporate between 2000 and 3000 individual 15 metre receiving dishes, which together can be expected to produce a data rate of many TB/s. Given such a high data rate, it becomes crucial to consider how this information will be processed and stored to maximise its scientific utility. In this paper, we consider one possible data processing scenario for the SKA, for the purposes of an all-sky pulsar survey. In particular we treat the selection of promising signals from the SKA processing pipeline as a data stream classification problem. We consider the feasibility of classifying signals that arrive via an unlabelled and heavily class imbalanced data stream, using currently available algorithms and frameworks. Our results indicate that existing stream learners exhibit unacceptably low recall on real astronomical data when used in standard configuration; however, good false positive performance and comparable accuracy to static learners, suggests they have definite potential as an on-line solution to this particular big data challenge.
6 Pages, 2 figures, to be published in Proceedings 2013 IEEE International Conference on Systems, Man, and Cybernetics (SMC)
References in corpus (3)
- The Parkes multibeam pulsar survey: VI. Discovery and timing of 142 pulsars and a Galactic population analysis
- Application of the Gaussian mixture model in pulsar astronomy -- pulsar classification and candidates ranking for {\it Fermi} 2FGL catalog
- Discovery of 28 pulsars using new techniques for sorting pulsar candidates
Cited by in corpus (4)
- Fifty Years of Pulsar Candidate Selection: From simple filters to a new principled real-time classification approach
- A Hybrid Ensemble method for Pulsar Candidate Classification
- Pulsars Detection by Machine Learning with Very Few Features
- Pulsar Candidate Sifting Using Multi-input Convolution Neural Networks