Photometric redshifts with machine learning, lights and shadows on a complex data science use case
arXiv:2104.06774 · doi:10.3389/fspas.2021.658229
Abstract
The current role of data-driven science is constantly increasing its importance within Astrophysics, due to the huge amount of multi-wavelength data collected every day, characterized by complex and high-volume information requiring efficient and as much as possible automated exploration tools. Furthermore, to accomplish main and legacy science objectives of future or incoming large and deep survey projects, such as JWST, LSST and Euclid, a crucial role is played by an accurate estimation of photometric redshifts, whose knowledge would permit the detection and analysis of extended and peculiar sources by disentangling low-z from high-z sources and would contribute to solve the modern cosmological discrepancies. The recent photometric redshift data challenges, organized within several survey projects, like LSST and Euclid, pushed the exploitation of multi-wavelength and multi-dimensional data observed or ad hoc simulated to improve and optimize the photometric redshifts prediction and statistical characterization based on both SED template fitting and machine learning methodologies. But they also provided a new impetus in the investigation on hybrid and deep learning techniques, aimed at conjugating the positive peculiarities of different methodologies, thus optimizing the estimation accuracy and maximizing the photometric range coverage, particularly important in the high-z regime, where the spectroscopic ground truth is poorly available. In such a context we summarize what learned and proposed in more than a decade of research.
arXiv commentPreprint version of the manuscript to appear in the Volume: "The Physical Challenges of Astrostatistics", special issue of "Frontiers in Astronomy and Space Sciences"
References in corpus (130)
- The Sloan Digital Sky Survey: Technical Summary
- The Wide-field Infrared Survey Explorer (WISE): Mission Description and Initial On-orbit Performance
- The physical properties of star forming galaxies in the low redshift universe
- The UKIRT Infrared Deep Sky Survey (UKIDSS)
- The Galaxy Evolution Explorer: A Space Ultraviolet Survey Mission
- The Cosmic Evolution Survey (COSMOS) -- Overview
- EAZY: A Fast, Public Photometric Redshift Code
- Accurate photometric redshifts for the CFHT Legacy Survey calibrated using the VIMOS VLT Deep Survey
- Bayesian photometric redshift estimation
- The COSMOS2015 Catalog: Exploring the 1<z<6 Universe with half a million galaxies
- Galaxy and Mass Assembly (GAMA): survey diagnostics and core data release
- The Hyper Suprime-Cam SSP Survey: Overview and Survey Design
- KiDS-450: Cosmological parameter constraints from tomographic weak gravitational lensing
- Importance Nested Sampling and the MultiNest Algorithm
- Measuring and Modelling the Redshift Evolution of Clustering: the Hubble Deep Field North
- LSST Science Book, Version 2.0
- Rotation-invariant convolutional neural networks for galaxy morphology prediction
- Galaxy Zoo 2: detailed morphological classifications for 304,122 galaxies from the Sloan Digital Sky Survey
- ANNz: estimating photometric redshifts using artificial neural networks
- A Modified Magnitude System that Produces Well-Behaved Magnitudes, Colors, and Errors Even for Low Signal-to-Noise Ratio Measurements
- Gravitational Lensing Analysis of the Kilo Degree Survey
- The Kilo-Degree Survey
- eROSITA Science Book: Mapping the Structure of the Energetic Universe
- The VIMOS VLT Deep Survey - First epoch VVDS-Deep survey: 11564 spectra with 17.5<=IAB<=24, and the redshift distribution over 0< z <=5
- Weak lensing for precision cosmology
- Photometric redshift and classification for the XMM-COSMOS sources
- The VISTA Deep Extragalactic Observations (VIDEO) Survey
- Photometric Redshifts for Hyper Suprime-Cam Subaru Strategic Program Data Release 1
- Slicing Through Multicolor Space: Galaxy Redshifts From Broadband Photometry
- CFHTLenS: Improving the quality of photometric redshifts with precision photometry
- The first and second data releases of the Kilo-Degree Survey
- Effect of Photometric Redshift Uncertainties on Weak Lensing Tomography
- TPZ : Photometric redshift PDFs and ancillary information by using prediction trees and random forests
- Dissecting Photometric redshift for Active Galactic Nuclei using XMM- and Chandra-COSMOS samples
- PHAT: PHoto-z Accuracy Testing
- The VIMOS Public Extragalactic Redshift Survey (VIPERS). Full spectroscopic data and auxiliary information release (PDR-2)
- The DEIMOS 10k spectroscopic survey catalog of the COSMOS field
- Photometric Redshifts for the SDSS Early Data Release
- The third data release of the Kilo-Degree Survey and associated data products
- Mapping the Galaxy Color-Redshift Relation: Optimal Photometric Redshift Calibration Strategies for Cosmology Surveys
- ANNz2 - photometric redshift and probability distribution function estimation using machine learning
- Estimating photometric redshifts with artificial neural networks
- Probabilistic Random Forest: A machine learning algorithm for noisy datasets
- Photometric redshifts from SDSS images using a Convolutional Neural Network
- CLASH-VLT: The mass, velocity-anisotropy, and pseudo-phase-space density profiles of the z=0.44 galaxy cluster MACS 1206.2-0847
- Photometric redshifts for the SDSS Data Release 12
- Dark matter halo properties of GAMA galaxy groups from 100 square degrees of KiDS weak lensing data
- Photometric redshift estimation via deep learning
- The Strong Gravitational Lens Finding Challenge
- Redshift distributions of galaxies in the DES Science Verification shear catalogue and implications for weak lensing
- Estimating the Redshift Distribution of Photometric Galaxy Samples II. Applications and Tests of a New Method
- Precision photometric redshift calibration for galaxy-galaxy weak lensing
- Machine Learning in Astronomy: a practical overview
- Photometric redshifts and quasar probabilities from a single, data-driven generative model
- Surveying the reach and maturity of machine learning and artificial intelligence in astronomy
- The weirdest SDSS galaxies: results from an outlier detection algorithm
- Wide-Field InfraRed Survey Telescope (WFIRST) Final Report
- Photometric redshifts with the Multilayer Perceptron Neural Network: application to the HDF-S and SDSS
- Cosmology with photometric redshift surveys
- Photometric Redshift Calibration with Self Organising Maps
- SOMz: photometric redshift PDFs with self organizing maps and random atlas
- Euclid preparation: X. The Euclid photometric-redshift challenge
- A Comparison of Six Photometric Redshift Methods Applied to 1.5 Million Luminous Red Galaxies
- The 31 Deg Release of the Stripe 82 X-ray Survey: The Point Source Catalog
- MUSE integral-field spectroscopy towards the Frontier Fields Cluster Abell S1063: I. Data products and redshift identifications
- Photometric redshifts for Quasars in multi band Surveys
- Photometric redshifts for the next generation of deep radio continuum surveys - I: Template fitting
- Star Formation Laws: the Effects of Gas Cloud Sampling
- Evaluation of probabilistic photometric redshift estimation approaches for The Rubin Observatory Legacy Survey of Space and Time (LSST)
- Using neural networks to estimate redshift distributions. An application to CFHTLenS
- Photometric redshifts for the Kilo-Degree Survey. Machine-learning analysis with artificial neural networks
- A Sparse Gaussian Process Framework for Photometric Redshift Estimation
- METAPHOR: A machine learning based method for the probability density estimation of photometric redshifts
- Photometric redshifts from reconstructed QSO templates
- Photometric Redshifts with the LSST: Evaluating Survey Observing Strategies
- The-wiZZ: Clustering redshift estimation for everyone
- AGN Populations in Large Volume X-ray Surveys: Photometric Redshifts and Population Types found in the Stripe 82X Survey
- The PAU Survey: Early demonstration of photometric redshift performance in the COSMOS field
- Deep learning Approach for Classifying, Detecting and Predicting Photometric Redshifts of Quasars in the Sloan Digital Sky Survey Stripe 82
- Finding Rare AGN: XMM-Newton and Chandra Observations of SDSS Stripe 82
- Evolution of galaxy size--stellar mass relation from the Kilo Degree Survey
- Anomaly detection for machine learning redshifts applied to SDSS galaxies
- A catalogue of photometric redshifts for the SDSS-DR9 galaxies
- Photometric Redshift Error Estimators
- Neural Networks and Photometric Redshifts
- Exhausting the Information: Novel Bayesian Combination of Photometric Redshift PDFs
- Star formation rates and stellar masses from machine learning
- Automated physical classification in the SDSS DR10. A catalogue of candidate Quasars
- Photometric classification of emission line galaxies with Machine Learning methods
- Photometric Redshift Probability Distributions for Galaxies in the SDSS DR8
- Incorporating Photometric Redshift Probability Density Information into Real-Space Clustering Measurements
- An analysis of feature relevance in the classification of astronomical transients with machine learning methods
- Mining the SDSS archive. I. Photometric redshifts in the nearby universe
- CPz: Classification-aided photometric-redshift estimation
- Machine Learning based photometric redshifts for the KiDS ESO DR2 galaxies
- A machine learning approach to galaxy properties: joint redshift-stellar mass probability distributions with Random Forest
- What Lies Beneath: Using p(z) to Reduce Systematic Photometric Redshift Errors
- Astroinformatics of galaxies and quasars: a new general method for photometric redshifts estimation
- A cooperative approach among methods for photometric redshifts estimation: an application to KiDS data
- Finding Rare AGN: X-ray Number Counts of Chandra Sources in Stripe 82
- Photometric redshifts with Quasi Newton Algorithm (MLPQNA). Results in the PHAT1 contest
- Photo-z Performance for Precision Cosmology
- Analysis of a Custom Support Vector Machine for Photometric Redshift Estimation and the Inclusion of Galaxy Shape Information
- Can Self-Organizing Maps accurately predict photometric redshifts?
- Return of the features. Efficient feature selection and interpretation for photometric redshifts
- Star Formation Rates for photometric samples of galaxies using machine learning methods
- Photometric redshift estimation using Gaussian processes
- Data-driven, interpretable photometric redshifts trained on heterogeneous and unrepresentative data
- Simulating JWST deep extragalactic imaging surveys and physical parameter recovery
- Weighing the Universe with Photometric Redshift Surveys and the Impact on Dark Energy Forecasts
- Improving the reliability of photometric redshift with machine learning
- The search for galaxy cluster members with deep learning of panchromatic HST imaging and extensive spectroscopy
- Approximating photo- PDFs for large surveys
- Photometric Redshifts based on standard SED fitting procedures
- Building the largest spectroscopic sample of ultra-compact massive galaxies with the Kilo Degree Survey
- Feature Selection Strategies for Classifying High Dimensional Astronomical Data Sets
- Statistical analysis of probability density functions for photometric redshifts through the KiDS-ESO-DR3 galaxies
- Photometric redshifts for X-ray-selected active galactic nuclei in the eROSITA era
- Characterizing and Propagating Modeling Uncertainties in Photometrically-Derived Redshift Distributions
- A Comparison of Photometric Redshift Techniques for Large Radio Surveys
- Euclid: The selection of quiescent and star-forming galaxies using observed colours
- The PAU Survey: narrowband photometric redshifts using Gaussian processes
- Sacrificing information for the greater good: how to select photometric bands for optimal accuracy
- QSO photometric redshifts from SDSS, WISE and GALEX colours
- Photometric Redshifts for the Hyper Suprime-Cam Subaru Strategic Program Data Release 2
- Nature versus nurture: relic nature and environment of the most massive passive galaxies at
- Weak Lensing Study in VOICE Survey I: Shear Measurement
- Mixture Models for Photometric Redshifts
- Photometric Redshifts and Photometry Errors
- Photometric Redshift Analysis using Supervised Learning Algorithms and Deep Learning
Cited by in corpus (12)
- Photometric Redshifts for Next-Generation Surveys
- Machine Learning for Observational Cosmology
- Stellar Populations With Optical Spectra: Deep Learning vs. Popular Spectrum Fitting Codes
- Extracting Photometric Redshift from Galaxy Flux and Image Data using Neural Networks in the CSST Survey
- Photometric redshift estimates using Bayesian neural networks in the CSST survey
- Estimating Photometric Redshifts for Galaxies from the DESI Legacy Imaging Surveys with Bayesian Neural Networks Trained by DESI EDR
- Augmenting photometric redshift estimates using spectroscopic nearest neighbours
- The Quasar Catalogue for S-PLUS DR4 (QuCatS) and the estimation of photometric redshifts
- Dark Energy Survey Deep Field photometric redshift performance and training incompleteness assessment
- Probing Cosmology beyond CDM using the SKA
- Improving Photometric Redshifts by Merging Probability Density Functions from Template-Based and Machine Learning Algorithms
- Impact of PSF misestimation and galaxy population bias on precision shear measurement using a CNN