Mapping the Galaxy Color-Redshift Relation: Optimal Photometric Redshift Calibration Strategies for Cosmology Surveys
arXiv:1509.03318 · doi:10.1088/0004-637X/813/1/53
Abstract
Calibrating the photometric redshifts of >10^9 galaxies for upcoming weak lensing cosmology experiments is a major challenge for the astrophysics community. The path to obtaining the required spectroscopic redshifts for training and calibration is daunting, given the anticipated depths of the surveys and the difficulty in obtaining secure redshifts for some faint galaxy populations. Here we present an analysis of the problem based on the self-organizing map, a method of mapping the distribution of data in a high-dimensional space and projecting it onto a lower-dimensional representation. We apply this method to existing photometric data from the COSMOS survey selected to approximate the anticipated Euclid weak lensing sample, enabling us to robustly map the empirical distribution of galaxies in the multidimensional color space defined by the expected Euclid filters. Mapping this multicolor distribution lets us determine where - in galaxy color space - redshifts from current spectroscopic surveys exist and where they are systematically missing. Crucially, the method lets us determine whether a spectroscopic training sample is representative of the full photometric space occupied by the galaxies in a survey. We explore optimal sampling techniques and estimate the additional spectroscopy needed to map out the color-redshift relation, finding that sampling the galaxy distribution in color space in a systematic way can efficiently meet the calibration requirements. While the analysis presented here focuses on the Euclid survey, similar analysis can be applied to other surveys facing the same calibration challenge, such as DES, LSST, and WFIRST.
ApJ accepted, 17 pages, 10 figures
References in corpus (13)
- The Cosmic Evolution Survey (COSMOS) -- Overview
- COSMOS Photometric Redshifts with 30-bands for 2-deg2
- An ultra-deep near-infrared spectrum of a compact quiescent galaxy at z=2.2
- zCOSMOS: A Large VLT/VIMOS redshift survey covering 0 < z < 3 in the COSMOS field
- UltraVISTA: a new ultra-deep near-infrared survey in COSMOS
- The VIMOS Ultra-Deep Survey: ~10,000 galaxies with spectroscopic redshifts to study galaxy assembly at early epochs 2<z<~6
- Calibrating Redshift Distributions Beyond Spectroscopic Limits with Cross-Correlations
- Optimal Surveys for Weak Lensing Tomography
- Magellan Spectroscopy of AGN Candidates in the COSMOS Field
- Clustering-based Redshift Estimation: Comparison to Spectroscopic Redshifts
- The GEEC2 spectroscopic survey of Galaxy Groups at
- Size of spectroscopic calibration samples for cosmic shear photometric redshifts
- Data augmentation for machine learning redshifts applied to SDSS galaxies
Cited by in corpus (141)
- KiDS-450: Cosmological parameter constraints from tomographic weak gravitational lensing
- Euclid. I. Overview of the Euclid mission
- Cosmology from cosmic shear power spectra with Subaru Hyper Suprime-Cam first-year data
- Dark Energy Survey Year 3 Results: Cosmology from Cosmic Shear and Robustness to Data Calibration
- Weak lensing for precision cosmology
- KiDS+VIKING-450: Cosmic shear tomography with optical+infrared data
- Photometric Redshifts for Hyper Suprime-Cam Subaru Strategic Program Data Release 1
- The CosmoVerse White Paper: Addressing observational tensions in cosmology with systematics and fundamental physics
- Dark Energy Survey Year 1 Results: Redshift distributions of the weak lensing source galaxies
- Dark Energy Survey Year 3 Results: Redshift Calibration of the Weak Lensing Source Galaxies
- KiDS+VIKING-450 and DES-Y1 combined: Cosmology with cosmic shear
- KiDS-1000 catalogue: Redshift distributions and their calibration
- Surveying the reach and maturity of machine learning and artificial intelligence in astronomy
- The Complete Calibration of the Color-Redshift Relation (C3R2) Survey: Survey Overview and Data Release 1
- Weak-lensing mass calibration of redMaPPer galaxy clusters in Dark Energy Survey Science Verification data
- The Complete Calibration of the Color-Redshift Relation (C3R2) Survey: Analysis and Data Release 2
- Photometric Redshift Calibration with Self Organising Maps
- GPz: Non-stationary sparse Gaussian processes for heteroscedastic uncertainty estimation in photometric redshifts
- Euclid preparation: X. The Euclid photometric-redshift challenge
- The COSMOS2015 galaxy stellar mass function: 13 billion years of stellar mass assembly in 10 snapshots
- Photometric Redshifts for Next-Generation Surveys
- Phenotypic redshifts with self-organizing maps: A novel method to characterize redshift distributions of source galaxies for weak lensing
- Morpheus: A Deep Learning Framework For Pixel-Level Analysis of Astronomical Image Data
- Source Selection for Cluster Weak Lensing Measurements in the Hyper Suprime-Cam Survey
- Evaluation of probabilistic photometric redshift estimation approaches for The Rubin Observatory Legacy Survey of Space and Time (LSST)
- Weak lensing Analysis of X-Ray-selected XXL Galaxy Groups and Clusters with Subaru HSC Data
- The eROSITA Final Equatorial-Depth Survey (eFEDS): X-ray Observable-to-Mass-and-Redshift Relations of Galaxy Clusters and Groups with Weak-Lensing Mass Calibration from the Hyper Suprime-Cam Subaru Strategic Program Survey
- METAPHOR: A machine learning based method for the probability density estimation of photometric redshifts
- Optimizing spectroscopic follow-up strategies for supernova photometric classification with active learning
- Photometric redshifts for the Kilo-Degree Survey. Machine-learning analysis with artificial neural networks
- The-wiZZ: Clustering redshift estimation for everyone
- Scientific Synergy Between LSST and Euclid
- Horizon-AGN virtual observatory - 1. SED-fitting performance and forecasts for future imaging surveys
- Redshift inference from the combination of galaxy colors and clustering in a hierarchical Bayesian model
- The PAU Survey: An improved photo- sample in the COSMOS field
- PS1-STRM: Neural network source classification and photometric redshift catalogue for PS1 DR1
- Selection biases in empirical p(z) methods for weak lensing
- Learning the Relationship between Galaxies Spectra and their Star Formation Histories using Convolutional Neural Networks and Cosmological Simulations
- Cosmology with dropout selection: Straw-man surveys and CMB lensing
- The impact of spectroscopic incompleteness in direct calibration of redshift distributions for weak lensing surveys
- KiDS & Euclid: Cosmological implications of a pseudo angular power spectrum analysis of KiDS-1000 cosmic shear tomography
- A review of unsupervised learning in astronomy
- Redshift inference from the combination of galaxy colors and clustering in a hierarchical Bayesian model Application to realistic -body simulations
- KiDS-1000: Cosmology with improved cosmic shear measurements
- A cooperative approach among methods for photometric redshifts estimation: an application to KiDS data
- Photometric Redshift Calibration Requirements for WFIRST Weak Lensing Cosmology: Predictions from CANDELS
- Horizon-AGN virtual observatory -- 2: Template-free estimates of galaxy properties from colours
- WFIRST Science Investigation Team "Cosmology with the High Latitude Survey" Annual Report 2017
- The PAU Survey: Photometric redshifts using transfer learning from simulations
- Catalog of quasars from the Kilo-Degree Survey Data Release 3
- Small near-Earth asteroids in the Palomar Transient Factory survey: A real-time streak-detection system
- Cosmology Intertwined: A Review of the Particle Physics, Astrophysics, and Cosmology Associated with the Cosmological Tensions and Anomalies
- A new method to assign galaxy cluster membership using photometric redshifts
- Spatial curvature endgame: Reaching the limit of curvature determination
- The MUSE Hubble Ultra Deep Field Survey: III. Testing photometric redshifts to 30th magnitude
- Photometric Redshifts from SDSS Images with an Interpretable Deep Capsule Network
- Dark Energy Survey Year 3 results: Marginalisation over redshift distribution uncertainties using ranking of discrete realisations
- KiDS-Legacy: Consistency of cosmic shear measurements and joint cosmological constraints with external probes
- Propagating sample variance uncertainties in redshift calibration: simulations, theory and application to the COSMOS2015 data
- Improving the reliability of photometric redshift with machine learning
- COSMOS2020: Manifold Learning to Estimate Physical Parameters in Large Galaxy Surveys
- Galaxy-Galaxy Lensing in HSC: Validation Tests and the Impact of Heterogeneous Spectroscopic Training Sets
- A Method to Distinguish Quiescent and Dusty Star-forming Galaxies with Machine Learning
- Photometric redshifts with machine learning, lights and shadows on a complex data science use case
- Combining the CLAUDS & HSC-SSP surveys: U+grizy(+YJHKs) photometry and photometric redshifts for 18M galaxies in the 20 deg2 of the HSC-SSP Deep and ultraDeep fields
- SED-dependent Galactic Extinction Prescription for Euclid and Future Cosmological Surveys
- Stacking for machine learning redshifts applied to SDSS galaxies
- Statistical analysis of probability density functions for photometric redshifts through the KiDS-ESO-DR3 galaxies
- HST Grism-derived Forecasts for Future Galaxy Redshift Surveys
- How to Find Variable Active Galactic Nuclei with Machine Learning
- Exploring Photometric Redshifts as an Optimization Problem: An Ensemble MCMC and Simulated Annealing-Driven Template-Fitting Approach
- Euclid preparation: VIII. The Complete Calibration of the Colour-Redshift Relation survey: VLT/KMOS observations and data release
- Extracting Photometric Redshift from Galaxy Flux and Image Data using Neural Networks in the CSST Survey
- A Comparison of Photometric Redshift Techniques for Large Radio Surveys
- Self-calibration and robust propagation of photometric redshift distribution uncertainties in weak gravitational lensing
- Comparison of Observed Galaxy Properties with Semianalytic Model Predictions using Machine Learning
- Genuine Retrieval of the AGN Host Stellar Population (GRAHSP)
- Augmenting machine learning photometric redshifts with Gaussian mixture models
- Using angular two-point correlations to self-calibrate the photometric redshift distributions of DECaLS DR9
- Universe opacity and CMB
- Photometric Redshift Estimation with Galaxy Morphology using Self-Organizing Maps
- Astrophysics and Big Data: Challenges, Methods, and Tools
- Exploring X-ray variability with unsupervised machine learning I. Self-organizing maps applied to XMM-Newton data
- Rainbow Cosmic Shear: Optimisation of Tomographic Bins
- A Machine Learning Approach to Predict Missing Flux Densities in Multi-band Galaxy Surveys
- PhotoWeb redshift: boosting photometric redshift accuracy with large spectroscopic surveys
- Euclid preparation: XX. The Complete Calibration of the Color-Redshift Relation survey: LBT observations and data release
- The 2-degree Field Lensing Survey: photometric redshifts from a large new training sample to r<19.5
- Wide Area VISTA Extra-galactic Survey (WAVES): Unsupervised star-galaxy separation on the WAVES-Wide photometric input catalogue using UMAP and
- Impact of stellar population synthesis choices on forward modelling-based redshift distribution estimates
- DESI Complete Calibration of the Color-Redshift Relation (DC3R2): Results from early DESI data
- Improving Photometric Redshift Estimates with Training Sample Augmentation
- pop-cosmos: Scaleable inference of galaxy properties and redshifts with a data-driven population model
- Constraining Gravity with a -cut Cosmic Shear Analysis of the Hyper Suprime-Cam First-Year Data
- PICZL: Image-based Photometric Redshifts for AGN
- ULISSE: A Tool for One-shot Sky Exploration and its Application to Active Galactic Nuclei Detection
- Self-consistent redshift estimation using correlation functions without a spectroscopic reference sample
- Data Deluge in Astrophysics: Photometric Redshifts as a Template Use Case
- SDSS-IV MaNGA: Unveiling Galaxy Interaction by Merger Stages with Machine Learning
- Rotation and flipping invariant self-organizing maps with astronomical images: A cookbook and application to the VLA Sky Survey QuickLook images
- Tuning target selection algorithms to improve galaxy redshift estimates
- Euclid preparation. XXXI. The effect of the variations in photometric passbands on photometric-redshift accuracy
- Improved Tomographic Binning of 3x2pt Lens Samples: Neural Network Classifiers and Optimal Bin Assignments
- Estimation of Photometric Redshifts. II. Identification of Out-of-Distribution Data with Neural Networks
- The many flavours of photometric redshifts
- Enhancing Photometric Redshift Catalogs Through Color-Space Analysis: Application to KiDS-Bright Galaxies
- Nonparametric Methods in Astronomy: Think, Regress, Observe -- Pick Any Three
- Improved photometric redshifts with colour-constrained galaxy templates for future wide-area surveys
- The End of Galaxy Surveys
- The Compilation and Validation of the Spectroscopic Redshift Catalogs for the DESI-COSMOS and DESI-XMMLSS Fields
- Hawaii Two-0: High-redshift galaxy clustering and bias
- Augmenting photometric redshift estimates using spectroscopic nearest neighbours
- Rejection criteria based on outliers in the KiDS photometric redshifts and PDF distributions derived by machine learning
- The optical luminosity function of LOFAR radio-selected quasars at in the NDWFS-Boötes field
- Improved Weak Lensing Photometric Redshift Calibration via StratLearn and Hierarchical Modeling
- Dark Energy Survey Year 3 Results: Deep Field Optical + Near-Infrared Images and Catalogue
- pop-cosmos: Insights from generative modeling of a deep, infrared-selected galaxy population
- Optimising the shape of photometric redshift distributions with clustering cross-correlations
- Galaxy Manifold: Characterizing and understanding galaxies with two parameters
- Bringing manifold learning and dimensionality reduction to SED fitters
- A Galaxy-Halo Model for Multiple Cosmological Tracers
- Euclid: Photometric redshift calibration with self-organising maps
- Optimizing Redshift Distribution Inference through Joint Self-Calibration and Clustering-Redshift Synergy
- Navigating AGN variability with self-organizing maps
- ULTIMATE-deblending I. A 50-band Ultraviolet to Mid-infrared Photometric Catalog Combining Space- and Ground-based Data in the JWST/PRIMER survey
- Euclid Preparation: XIV. The Complete Calibration of the Color-Redshift Relation (C3R2) Survey: Data Release 3
- Euclid: Constraining ensemble photometric redshift distributions with stacked spectroscopy
- Reconstructing redshift distributions with photometric galaxy clustering
- Lensing Without Borders. I. A Blind Comparison of the Amplitude of Galaxy-Galaxy Lensing Between Independent Imaging Surveys
- SHAPE. I. A SOM-SED hybrid approach for efficient galaxy parameter estimation leveraging JWST
- Dusty Clump Survival in Supernova Ejecta: Dust-Mediated Growth vs. Crushing by the Reverse Shock
- ULISSE: Determination of star-formation rate and stellar mass based on the one-shot galaxy imaging technique
- Probability density estimation of photometric redshifts based on machine learning
- Deriving Photometric Redshifts using Fuzzy Archetypes and Self-Organizing Maps. II. Comparing Sampling Techniques Using Mock Data
- Identifying and Repairing Catastrophic Errors in Galaxy Properties Using Dimensionality Reduction
- Identifying AGN host galaxies by Machine Learning with HSC+WISE
- Euclid: Early Release Observations. Weak gravitational lensing analysis of Abell 2390
- Categorizing models using Self-Organizing Maps: an application to modified gravity theories probed by cosmic shear
- GalSBI-SPS: a stellar population synthesis-based galaxy population model for cosmology and galaxy evolution applications
- The Redshifts from 122 Bands: Comparative Redshift Forecast for Low-Resolution Spectra from SPHEREx and 7-Dimensional Sky Survey (7DS)
- Euclid: Improving redshift distribution reconstruction using a deep-to-wide transfer function