Robust Machine Learning Applied to Astronomical Datasets III: Probabilistic Photometric Redshifts for Galaxies and Quasars in the SDSS and GALEX

Abstract

We apply machine learning in the form of a nearest neighbor instance-based algorithm (NN) to generate full photometric redshift probability density functions (PDFs) for objects in the Fifth Data Release of the Sloan Digital Sky Survey (SDSS DR5). We use a conceptually simple but novel application of NN to generate the PDFs - perturbing the object colors by their measurement error - and using the resulting instances of nearest neighbor distributions to generate numerous individual redshifts. When the redshifts are compared to existing SDSS spectroscopic data, we find that the mean value of each PDF has a dispersion between the photometric and spectroscopic redshift consistent with other machine learning techniques, being sigma = 0.0207 +/- 0.0001 for main sample galaxies to r < 17.77 mag, sigma = 0.0243 +/- 0.0002 for luminous red galaxies to r < ~19.2 mag, and sigma = 0.343 +/- 0.005 for quasars to i < 20.3 mag. The PDFs allow the selection of subsets with improved statistics. For quasars, the improvement is dramatic: for those with a single peak in their probability distribution, the dispersion is reduced from 0.343 to sigma = 0.117 +/- 0.010, and the photometric redshift is within 0.3 of the spectroscopic redshift for 99.3 +/- 0.1% of the objects. Thus, for this optical quasar sample, we can virtually eliminate 'catastrophic' photometric redshift estimates. In addition to the SDSS sample, we incorporate ultraviolet photometry from the Third Data Release of the Galaxy Evolution Explorer All-Sky Imaging Survey (GALEX AIS GR3) to create PDFs for objects seen in both surveys. For quasars, the increased coverage of the observed frame UV of the SED results in significant improvement over the full SDSS sample, with sigma = 0.234 +/- 0.010. We demonstrate that this improvement is genuine. [Abridged]

Accepted to ApJ, 10 pages, 12 figures, uses emulateapj.cls

References in corpus (44)

• Maps of Dust IR Emission for Use in Estimation of Reddening and CMBR Foregrounds
• The Sloan Digital Sky Survey: Technical Summary
• The Galaxy Evolution Explorer: A Space Ultraviolet Survey Mission
• Spectroscopic Target Selection in the Sloan Digital Sky Survey: The Main Galaxy Sample
• Composite Quasar Spectra From the Sloan Digital Sky Survey
• Bayesian photometric redshift estimation
• Spectroscopic Target Selection for the Sloan Digital Sky Survey: The Luminous Red Galaxy Sample
• Spectroscopic Target Selection in the Sloan Digital Sky Survey: The Quasar Sample
• The SDSS Quasar Survey: Quasar Luminosity Function from Data Release Three
• The Fifth Data Release of the Sloan Digital Sky Survey
• ANNz: estimating photometric redshifts using artificial neural networks
• A catalogue of the Chandra Deep Field South with multi-colour classification and photometric redshifts from COMBO-17
• A new catalog of photometric redshifts in the Hubble Deep Field
• Photometric Redshifts for the SDSS Early Data Release
• Star-forming galaxies at very high redshifts
• Estimating photometric redshifts with artificial neural networks
• Spectral Classification of Galaxies: An Orthogonal Approach
• Distributions of Galaxy Spectral Types in the Sloan Digital Sky Survey
• A Galaxy Photometric Redshift Catalog for the Sloan Digital Sky Survey Data Release 6
• Photometric Redshifts in the IRAC Shallow Survey
• Galaxy Types in the Sloan Digital Sky Survey Using Supervised Artificial Neural Networks
• Photometric redshifts with the Multilayer Perceptron Neural Network: application to the HDF-S and SDSS
• Estimating Photometric Redshifts Using Support Vector Machines
• Cosmology with photometric redshift surveys
• Photometric Redshifts of Quasars
• MegaZ-LRG: A photometric redshift catalogue of one million SDSS Luminous Red Galaxies
• Robust Machine Learning Applied to Astronomical Datasets I: Star-Galaxy Classification of the SDSS DR3 Using Decision Trees
• A Robust Classification of Galaxy Spectra: Dealing with Noisy and Incomplete Data
• An Empirical Algorithm for Broad-band Photometric Redshifts of Quasars from the Sloan Digital Sky Survey
• A Bayesian Classifier for Photometric Redshifts: Identification of high redshift clusters
• Novel Methods for Predicting Photometric Redshifts from Broad Band Photometry using Virtual Sensors
• Photometric redshifts from reconstructed QSO templates
• Towards More Precise Photometric Redshifts: Calibration Via CCD Photometry
• Robust Machine Learning Applied to Astronomical Datasets II: Quantifying Photometric Redshifts for Quasars Using Instance-Based Learning
• Measuring our universe from galaxy redshift surveys
• Mining the SDSS archive. I. Photometric redshifts in the nearby universe
• Empirical Photometric Redshifts of Luminous Red Galaxies and Clusters in SDSS
• Color-redshift Relations and Photometric Redshift Estimations of Quasars in Large Sky Surveys
• Two novel approaches for photometric redshift estimation based on SDSS and 2MASS databases
• $\mu$--PhotoZ: Photometric Redshifts by Inverting the Tolman Surface Brightness Test
• A New Technique for Galaxy Photometric Redshifts in the Sloan Digital Sky Survey
• Survey Requirements for Accurate and Precise Photometric Redshifts for Type Ia Supernovae
• Wide Field Surveys and Astronomical Discovery Space
• Sky surveys and deep fields of ground-based and space telescopes

Cited by in corpus (63)

• Data Mining and Machine Learning in Astronomy
• Fitting the integrated Spectral Energy Distributions of Galaxies
• TPZ : Photometric redshift PDFs and ancillary information by using prediction trees and random forests
• Star-galaxy Classification Using Deep Convolutional Neural Networks
• Photometric redshifts for the CFHTLS T0004 Deep and Wide fields
• Models and Simulations for the Photometric LSST Astronomical Time Series Classification Challenge (PLAsTiCC)
• Photometric redshifts and quasar probabilities from a single, data-driven generative model
• ArborZ: Photometric Redshifts Using Boosted Decision Trees
• DNF - Galaxy photometric redshift by Directional Neighbourhood Fitting
• SOMz: photometric redshift PDFs with self organizing maps and random atlas
• GPz: Non-stationary sparse Gaussian processes for heteroscedastic uncertainty estimation in photometric redshifts
• The Field X-ray AGN Fraction to z=0.7 from the Chandra Multiwavelength Project and the Sloan Digital Sky Survey
• Photometric redshifts for Quasars in multi band Surveys
• Photometric redshifts for the next generation of deep radio continuum surveys - I: Template fitting
• Shedding Light on the Galaxy Luminosity Function
• A Sparse Gaussian Process Framework for Photometric Redshift Estimation
• Photometric Redshifts with the LSST: Evaluating Survey Observing Strategies
• Ganalyzer: A tool for automatic galaxy image analysis
• Evolution of faint radio sources in the VIDEO-XMM3 field
• Sloan Digital Sky Survey III Photometric Quasar Clustering: Probing the Initial Conditions of the Universe using the Largest Volume
• Quasar Probabilities and Redshifts from WISE mid-IR through GALEX UV Photometry
• Exhausting the Information: Novel Bayesian Combination of Photometric Redshift PDFs
• Deep Transfer Learning for Star Cluster Classification: I. Application to the PHANGS-HST Survey
• New Approaches To Photometric Redshift Prediction Via Gaussian Process Regression In The Sloan Digital Sky Survey
• Incorporating Photometric Redshift Probability Density Information into Real-Space Clustering Measurements
• Astroinformatics of galaxies and quasars: a new general method for photometric redshifts estimation
• Bayesian photometric redshifts with empirical training sets
• Photometric Redshift Estimation Using Spectral Connectivity Analysis
• XMMPZCAT: A catalogue of photometric redshifts for X-ray sources
• Return of the features. Efficient feature selection and interpretation for photometric redshifts
• Star Formation Rates for photometric samples of galaxies using machine learning methods
• The ILIUM forward modelling algorithm for multivariate parameter estimation and its application to derive stellar parameters from Gaia spectrophotometry
• Astrometric Redshifts for Quasars
• Predicting star formation properties of galaxies using deep learning
• Photometric Redshifts from SDSS Images with an Interpretable Deep Capsule Network
• The Next Generation Virgo Cluster Survey. XV. The photometric redshift estimation for background sources
• Improving the reliability of photometric redshift with machine learning
• Estimating Photometric Redshifts of Quasars via K-nearest Neighbor Approach Based on Large Survey Databases
• Assessing the photometric redshift precision of the S-PLUS survey: the Stripe-82 as a test-case
• Statistical analysis of probability density functions for photometric redshifts through the KiDS-ESO-DR3 galaxies
• Use of neural networks for the identification of new z>=3.6 QSOs from FIRST-SDSS DR5
• Quantitative analysis of spirality in elliptical galaxies
• Comparison of Observed Galaxy Properties with Semianalytic Model Predictions using Machine Learning
• QSO photometric redshifts using machine learning and neural networks
• IRAC Near-Infrared Features in the Outer Parts of S4G Galaxies
• PhotoRedshift-MML: a multimodal machine learning method for estimating photometric redshifts of quasars
• Predicting the redshift of gamma-ray loud AGNs using supervised machine learning
• Photometric Redshift Estimation for Quasars by Integration of KNN and SVM
• Incorporating Measurement Error in Astronomical Object Classification
• QSO photometric redshifts from SDSS, WISE and GALEX colours
• Using Colors to Improve Photometric Metallicity Estimates for Galaxies
• GeneticKNN: A Weighted KNN Approach Supported by Genetic Algorithm for Photometric Redshift Estimation of Quasars
• Estimation of Photometric Redshifts. I. Machine Learning Inference for Pan-STARRS1 Galaxies Using Neural Networks
• Selection of powerful radio galaxies with machine learning
• QSO redshift estimates from optical, near-infrared and ultraviolet colours
• Euclid: Identifying the reddest high-redshift galaxies in the Euclid Deep Fields with gradient-boosted trees
• Degradation analysis in the estimation of photometric redshifts from non-representative training sets
• The Quasar Catalogue for S-PLUS DR4 (QuCatS) and the estimation of photometric redshifts
• Rejection criteria based on outliers in the KiDS photometric redshifts and PDF distributions derived by machine learning
• LinCDE: Conditional Density Estimation via Lindsey's Method
• Robust Machine Learning Applied to Terascale Astronomical Datasets
• Machine learning techniques for astrophysical modelling and photometric redshift estimation of quasars in optical sky surveys
• Robust Machine Learning Applied to Terascale Astronomical Datasets