Unsupervised self-organised mapping: a versatile empirical tool for object selection, classification and redshift estimation in large surveys

Abstract

We present an application of unsupervised machine learning - the self-organised map (SOM) - as a tool for visualising, exploring and mining the catalogues of large astronomical surveys. Self-organisation culminates in a low-resolution representation of the 'topology' of a parameter volume, and this can be exploited in various ways pertinent to astronomy. Using data from the Cosmological Evolution Survey (COSMOS), we demonstrate two key astronomical applications of the SOM: (i) object classification and selection, using the example of galaxies with active galactic nuclei as a demonstration, and (ii) photometric redshift estimation, illustrating how SOMs can be used as totally empirical predictive tools. With a training set of ~3800 galaxies with z_spec<1, we achieve photometric redshift accuracies competitive with other (mainly template fitting) techniques that use a similar number of photometric bands (sigma(Dz)=0.03 with a ~2% outlier rate when using u*-band to 8um photometry). We also test the SOM as a photo-z tool using the PHoto-z Accuracy Testing (PHAT) synthetic catalogue of Hildebrandt et al. (2010), which compares several different photo-z codes using a common input/training set. We find that the SOM can deliver accuracies that are competitive with many of the established template-fitting and empirical methods. This technique is not without clear limitations, which are discussed, but we suggest it could be a powerful tool in the era of extremely large - 'petabyte' - databases where efficient data-mining is a paramount concern.

13 pages, 6 figures, accepted for publication in MNRAS

References in corpus (7)

• The Cosmic Evolution Survey (COSMOS) -- Overview
• COSMOS Photometric Redshifts with 30-bands for 2-deg2
• zCOSMOS: A Large VLT/VIMOS redshift survey covering 0 < z < 3 in the COSMOS field
• EMU: Evolutionary Map of the Universe
• The Zurich Extragalactic Bayesian Redshift Analyzer (ZEBRA) and its first application: COSMOS
• Photometric Redshifts of Galaxies in COSMOS
• A large population of mid-infrared selected, obscured active galaxies in the Bootes field

Cited by in corpus (53)

• TPZ : Photometric redshift PDFs and ancillary information by using prediction trees and random forests
• 2MASS Photometric Redshift catalog: a comprehensive three-dimensional census of the whole sky
• Mapping the Galaxy Color-Redshift Relation: Optimal Photometric Redshift Calibration Strategies for Cosmology Surveys
• ANNz2 - photometric redshift and probability distribution function estimation using machine learning
• SOMz: photometric redshift PDFs with self organizing maps and random atlas
• GPz: Non-stationary sparse Gaussian processes for heteroscedastic uncertainty estimation in photometric redshifts
• Phenotypic redshifts with self-organizing maps: A novel method to characterize redshift distributions of source galaxies for weak lensing
• An automatic taxonomy of galaxy morphology using unsupervised machine learning
• Identifying Strong Lenses with Unsupervised Machine Learning using Convolutional Autoencoder
• A Sparse Gaussian Process Framework for Photometric Redshift Estimation
• Using neural networks to estimate redshift distributions. An application to CFHTLenS
• Anomaly detection for machine learning redshifts applied to SDSS galaxies
• Beyond the Hubble Sequence -- Exploring Galaxy Morphology with Unsupervised Machine Learning
• StarGO: A New Method to Identify the Galactic Origins of Halo Stars
• Exhausting the Information: Novel Bayesian Combination of Photometric Redshift PDFs
• Unsupervised star, galaxy, qso classification: Application of HDBSCAN
• Discovering the Unexpected in Astronomical Survey Data
• Cataloging the radio-sky with unsupervised machine learning: a new approach for the SKA era
• Radio Galaxy Zoo: Unsupervised Clustering of Convolutionally Auto-encoded Radio-astronomical Images
• Composite Spectral Energy Distributions and Infrared-Optical Colors of Type 1 and Type 2 Quasars
• Order out of Randomness : Self-Organization Processes in Astrophysics
• AstroVaDEr: Astronomical Variational Deep Embedder for Unsupervised Morphological Classification of Galaxies and Synthetic Image Generation
• A machine learning approach to galaxy properties: joint redshift-stellar mass probability distributions with Random Forest
• A review of unsupervised learning in astronomy
• Horizon-AGN virtual observatory -- 2: Template-free estimates of galaxy properties from colours
• Exploring the spectroscopic diversity of type Ia supernovae with DRACULA: a machine learning approach
• Machine Learning for Observational Cosmology
• Photometric Redshifts from SDSS Images with an Interpretable Deep Capsule Network
• Improving the reliability of photometric redshift with machine learning
• Estimating Photometric Redshifts of Quasars via K-nearest Neighbor Approach Based on Large Survey Databases
• COSMOS2020: Manifold Learning to Estimate Physical Parameters in Large Galaxy Surveys
• Radio Galaxy Zoo: Knowledge Transfer Using Rotationally Invariant Self-Organising Maps
• Self-Organizing Systems in Planetary Physics: Harmonic Resonances of Planet and Moon Orbits
• Organised Randoms: Learning and correcting for systematic galaxy clustering patterns in KiDS using self-organising maps
• Discovery of Peculiar Radio Morphologies with ASKAP using Unsupervised Machine Learning
• A Comparison of Photometric Redshift Techniques for Large Radio Surveys
• Rapid sorting of radio galaxy morphology using Haralick features
• Sacrificing information for the greater good: how to select photometric bands for optimal accuracy
• Photometric Redshift Estimation with Galaxy Morphology using Self-Organizing Maps
• Exploring X-ray variability with unsupervised machine learning I. Self-organizing maps applied to XMM-Newton data
• ASPECT: A spectra clustering tool for exploration of large spectral surveys
• Wide Area VISTA Extra-galactic Survey (WAVES): Unsupervised star-galaxy separation on the WAVES-Wide photometric input catalogue using UMAP and $\mathrm{HDBSCAN}$
• Classification algorithms applied to structure formation simulations
• CLaSPS: a new methodology for Knowledge extraction from complex astronomical dataset
• Identifying new high-confidence polluted white dwarf candidates using Gaia XP spectra and Self-Organizing Maps
• On constraining Cosmology and the Halo Mass Function with Weak Gravitational Lensing
• The PAU Survey: Photometric redshift estimation in deep wide fields
• Merged or monolithic? Using machine-learning to reconstruct the dynamical history of simulated star clusters
• Euclid: Photometric redshift calibration with self-organising maps
• YOUNG Star detrending for Transiting Exoplanet Recovery (YOUNGSTER) II: Using Self-Organising Maps to explore young star variability in Sectors 1-13 of TESS data
• Astroinformatics Challenges from Next-generation Radio Continuum Surveys
• Automated quasar continuum estimation using neural networks: a comparative study of deep-learning architectures
• SOMBI: Bayesian identification of parameter relations in unstructured cosmological data