Photometric Redshift Calibration with Self Organising Maps
arXiv:1909.09632 · doi:10.1051/0004-6361/201936782
Abstract
Accurate photometric redshift calibration is central to the robustness of all cosmology constraints from cosmic shear surveys. Analyses of the KiDS re-weighted training samples from all overlapping spectroscopic surveys to provide a direct redshift calibration. Using self-organising maps (SOMs) we demonstrate that this spectroscopic compilation is sufficiently complete for KiDS, representing of the effective 2D cosmic shear sample. We use the SOM to define a represented `gold' cosmic shear sample, per tomographic bin. Using mock simulations of KiDS and the spectroscopic training set, we estimate the uncertainty on the SOM redshift calibration, and find that photometric noise, sample variance, and spectroscopic selection effects (including redshift and magnitude incompleteness) induce a combined maximal scatter on the bias of the redshift distribution reconstruction () of in all tomographic bins. We show that the SOM calibration is unbiased in the cases of noiseless photometry and perfectly representative spectroscopic datasets, as expected from theory. The inclusion of both photometric noise and spectroscopic selection effects in our mock data introduces a maximal bias of , or at confidence, once quality flags have been applied to the SOM. The method presented here represents a significant improvement over the previously adopted direct redshift calibration implementation for KiDS, owing to its diagnostic and quality assurance capabilities. The implementation of this method in future cosmic shear studies will allow better diagnosis, examination, and mitigation of systematic biases in photometric redshift calibration.
arXiv comment22 pages, 10 figures, 4 appendices, accepted for publication in A&A
References in corpus (48)
- The COSMOS2015 Catalog: Exploring the 1<z<6 Universe with half a million galaxies
- KiDS-450: Cosmological parameter constraints from tomographic weak gravitational lensing
- Cosmology and Fundamental Physics with the Euclid Satellite
- UltraVISTA: a new ultra-deep near-infrared survey in COSMOS
- The DEEP2 Galaxy Redshift Survey: Design, Observations, Data Reduction, and Redshifts
- CFHTLenS: The Canada-France-Hawaii Telescope Lensing Survey
- Dark Energy Survey Year 1 Results: Cosmological Constraints from Cosmic Shear
- Cosmology from cosmic shear power spectra with Subaru Hyper Suprime-Cam first-year data
- Gravitational Lensing Analysis of the Kilo Degree Survey
- The VIMOS VLT Deep Survey final data release: a spectroscopic sample of 35016 galaxies and AGN out to z~6.7 selected with 17.5<=i_{AB}<=24.7
- KiDS+VIKING-450: Cosmic shear tomography with optical+infrared data
- The Great Observatories Origins Deep Survey. VLT/FORS2 Spectroscopy in the GOODS-South Field: Part III
- Photometric Redshifts for Hyper Suprime-Cam Subaru Strategic Program Data Release 1
- The fourth data release of the Kilo-Degree Survey: ugri imaging and nine-band optical-IR photometry over 1000 square degrees
- CFHTLenS: Improving the quality of photometric redshifts with precision photometry
- Calibrating Redshift Distributions Beyond Spectroscopic Limits with Cross-Correlations
- The Great Observatories Origins Deep Survey - VLT/VIMOS Spectroscopy in the GOODS-South Field: Part II
- The Great Observatories Origins Deep Survey. VLT/VIMOS Spectroscopy in the GOODS-South Field
- Bayesian Galaxy Shape Measurement for Weak Lensing Surveys -I. Methodology and a Fast Fitting Algorithm
- The MICE Grand Challenge Lightcone Simulation I: Dark matter clustering
- The third data release of the Kilo-Degree Survey and associated data products
- Estimating the Redshift Distribution of Faint Galaxy Samples
- Mapping the Galaxy Color-Redshift Relation: Optimal Photometric Redshift Calibration Strategies for Cosmology Surveys
- Dark Energy Survey Year 1 Results: Redshift distributions of the weak lensing source galaxies
- KiDS-450: Testing extensions to the standard cosmological model
- The MICE Grand Challenge Lightcone Simulation III: Galaxy lensing mocks from all-sky lensing maps
- The MICE Grand Challenge Lightcone Simulation II: Halo and Galaxy catalogues
- The WIRCam Deep Survey I: Counts, colours and mass-functions derived from near-infrared imaging in the CFHTLS deep fields
- An algorithm to build mock galaxy catalogues using MICE simulations
- KiDS+VIKING-450 and DES-Y1 combined: Cosmology with cosmic shear
- KiDS+VIKING-450 and DES-Y1 combined: Mitigating baryon feedback uncertainty with COSEBIs
- The Complete Calibration of the Color-Redshift Relation (C3R2) Survey: Survey Overview and Data Release 1
- The Complete Calibration of the Color-Redshift Relation (C3R2) Survey: Analysis and Data Release 2
- CosmoDC2: A Synthetic Sky Catalog for Dark Energy Science with LSST
- On using angular cross-correlations to determine source redshift distributions
- KiDS+VIKING-450: A new combined optical & near-IR dataset for cosmology and astrophysics
- The Large Early Galaxy Astrophysics Census (LEGA-C) Data Release II: dynamical and stellar population properties of z ~< 1 galaxies in the COSMOS field
- Phenotypic redshifts with self-organizing maps: A novel method to characterize redshift distributions of source galaxies for weak lensing
- Survey geometry and the internal consistency of recent cosmic shear measurements
- Using Galaxy Two-point Correlation Functions to Determine the Redshift Distributions of Galaxies Binned by Photometric Redshift
- First discoveries of z~6 quasars with the Kilo Degree Survey and VISTA Kilo-Degree Infrared Galaxy survey
- RCSLenS: The Red Cluster Sequence Lensing Survey
- Measuring the growth of matter fluctuations with third-order galaxy correlations
- The-wiZZ: Clustering redshift estimation for everyone
- Horizon-AGN virtual observatory - 1. SED-fitting performance and forecasts for future imaging surveys
- GaaP: PSF- and aperture-matched photometry using shapelets
- Horizon-AGN virtual observatory -- 2: Template-free estimates of galaxy properties from colours
- Galaxy Tagging: photometric redshift refinement and group richness enhancement
Cited by in corpus (90)
- KiDS-1000 Cosmology: Cosmic shear constraints and comparison between two point statistics
- KiDS-1000 Cosmology: Multi-probe weak gravitational lensing and spectroscopic galaxy clustering constraints
- The CosmoVerse White Paper: Addressing observational tensions in cosmology with systematics and fundamental physics
- Dark Energy Survey Year 3 Results: Redshift Calibration of the Weak Lensing Source Galaxies
- KiDS+VIKING-450 and DES-Y1 combined: Cosmology with cosmic shear
- KiDS-1000 catalogue: Redshift distributions and their calibration
- KiDS-1000 catalogue: Weak gravitational lensing shear measurements
- KiDS-1000 Cosmology: constraints beyond flat CDM
- KiDS-1000 Methodology: Modelling and inference for joint weak gravitational lensing and spectroscopic galaxy clustering analysis
- The SRG/eROSITA all-sky survey: Cosmology constraints from cluster abundances in the western Galactic hemisphere
- AbacusHOD: A highly efficient extended multi-tracer HOD framework and its application to BOSS and eBOSS data
- Consistent lensing and clustering in a low- Universe with BOSS, DES Year 3, HSC Year 1 and KiDS-1000
- The growth of density perturbations in the last 10 billion years from tomographic large-scale structure data
- Joint constraints on cosmology and the impact of baryon feedback: combining KiDS-1000 lensing with the thermal Sunyaev-Zeldovich effect from Planck and ACT
- KiDS+VIKING-450: Improved cosmological parameter constraints from redshift calibration with self-organising maps
- KiDS-1000: cosmic shear with enhanced redshift calibration
- Cosmic shear power spectra in practice
- Testing KiDS cross-correlation redshifts with simulations
- Indefinitely Flat Circular Velocities and the Baryonic Tully-Fisher Relation from Weak Lensing
- A Full CDM Analysis of KiDS-1000 Weak Lensing Maps using Deep Learning
- KiDS & Euclid: Cosmological implications of a pseudo angular power spectrum analysis of KiDS-1000 cosmic shear tomography
- Strong detection of the CMB lensingxgalaxy weak lensingcross-correlation from ACT-DR4,PlanckLegacy and KiDS-1000
- A review of unsupervised learning in astronomy
- Cosmic shear with small scales: DES-Y3, KiDS-1000 and HSC-DR1
- Redshift inference from the combination of galaxy colors and clustering in a hierarchical Bayesian model Application to realistic -body simulations
- KiDS-Legacy calibration: unifying shear and redshift calibration with the SKiLLS multi-band image simulations
- Cosmology Intertwined: A Review of the Particle Physics, Astrophysics, and Cosmology Associated with the Cosmological Tensions and Anomalies
- Constraints on from a full-scale and full-shape analysis of redshift-space clustering and galaxy-galaxy lensing in BOSS
- Magnification bias in galaxy surveys with complex sample selection functions
- Photometric Redshifts from SDSS Images with an Interpretable Deep Capsule Network
- KiDS-Legacy: Consistency of cosmic shear measurements and joint cosmological constraints with external probes
- Forward modeling of galaxy populations for cosmological redshift distribution inference
- Propagating sample variance uncertainties in redshift calibration: simulations, theory and application to the COSMOS2015 data
- KiDS-SBI: Simulation-based inference analysis of KiDS-1000 cosmic shear
- Photometric redshifts with machine learning, lights and shadows on a complex data science use case
- Probing galaxy bias and intergalactic gas pressure with KiDS Galaxies-tSZ-CMB lensing cross-correlations
- Radial acceleration relation of galaxies with joint kinematic and weak-lensing data
- Discovery of Peculiar Radio Morphologies with ASKAP using Unsupervised Machine Learning
- Organised Randoms: Learning and correcting for systematic galaxy clustering patterns in KiDS using self-organising maps
- Euclid preparation: VIII. The Complete Calibration of the Colour-Redshift Relation survey: VLT/KMOS observations and data release
- Self-calibration and robust propagation of photometric redshift distribution uncertainties in weak gravitational lensing
- Comparison of Observed Galaxy Properties with Semianalytic Model Predictions using Machine Learning
- KiDS-1000 Cosmology: machine learning -- accelerated constraints on Interacting Dark Energy with COSMOPOWER
- Using angular two-point correlations to self-calibrate the photometric redshift distributions of DECaLS DR9
- Analytic marginalization of uncertainties in tomographic galaxy surveys
- The SRG/eROSITA All-Sky Survey : Constraints on the structure growth from cluster number counts
- Analytical marginalisation over photometric redshift uncertainties in cosmic shear analyses
- The SRG/eROSITA All-Sky Survey: Weak-Lensing of eRASS1 Galaxy Clusters in KiDS-1000 and Consistency Checks with DES Y3 & HSC-Y3
- The SRG-eROSITA All-Sky Survey : Constraints on f(R) Gravity from Cluster Abundance
- TOPz: Photometric redshifts for J-PAS
- A roadmap to cosmological parameter analysis with third-order shear statistics III: Efficient estimation of third-order shear correlation functions and an application to the KiDS-1000 data
- Simulation-based inference of deep fields: galaxy population model and redshift distributions
- Impact of stellar population synthesis choices on forward modelling-based redshift distribution estimates
- Wide Area VISTA Extra-galactic Survey (WAVES): Unsupervised star-galaxy separation on the WAVES-Wide photometric input catalogue using UMAP and
- Euclid: Calibrating photometric redshifts with spectroscopic cross-correlations
- KiDS+VIKING-450: An internal-consistency test for cosmic shear tomography with a colour-based split of source galaxies
- pop-cosmos: Scaleable inference of galaxy properties and redshifts with a data-driven population model
- KiDS-Legacy: Covariance validation and the unified OneCovariance framework for projected large-scale structure observables
- Rotation and flipping invariant self-organizing maps with astronomical images: A cookbook and application to the VLA Sky Survey QuickLook images
- CLMM: a LSST-DESC Cluster weak Lensing Mass Modeling library for cosmology
- Euclid preparation. XXXI. The effect of the variations in photometric passbands on photometric-redshift accuracy
- Improved Tomographic Binning of 3x2pt Lens Samples: Neural Network Classifiers and Optimal Bin Assignments
- KiDS-Legacy: Angular galaxy clustering from deep surveys with complex selection effects
- Cosmic star formation history with tomographic cosmic infrared background-galaxy cross-correlation
- Enhancing Photometric Redshift Catalogs Through Color-Space Analysis: Application to KiDS-Bright Galaxies
- Combining cosmic shear data with correlated photo- uncertainties: constraints from DESY1 and HSC-DR1
- The dark matter halo masses of elliptical galaxies as a function of observationally robust quantities
- AMICO galaxy clusters in KiDS-1000: cosmological constraints and mass calibration from counts and weak lensing
- Euclid: Photometric redshift calibration with the clustering redshifts technique
- Constraining Hot Dark Matter Sub-Species with Weak Lensing and the Cosmic Microwave Background Radiation
- Improved Weak Lensing Photometric Redshift Calibration via StratLearn and Hierarchical Modeling
- YOUNG Star detrending for Transiting Exoplanet Recovery (YOUNGSTER) II: Using Self-Organising Maps to explore young star variability in Sectors 1-13 of TESS data
- Optimising the shape of photometric redshift distributions with clustering cross-correlations
- Euclid: Photometric redshift calibration with self-organising maps
- KiDS-1000: Detection of deviations from a purely cold dark matter power spectrum with tomographic weak gravitational lensing
- Optimizing Redshift Distribution Inference through Joint Self-Calibration and Clustering-Redshift Synergy
- Mapping Variations of Redshift Distributions with Probability Integral Transforms
- Disentangling the Halo: Joint Model for Measurements of the Kinetic Sunyaev-Zeldovich Effect and Galaxy-Galaxy Lensing
- 6x2pt: Forecasting gains from joint weak lensing and galaxy clustering analyses with spectroscopic-photometric galaxy cross-correlations
- The SRG/eROSITA all-sky survey: Constraints on Ultra-light Axion Dark Matter through Galaxy Cluster Number Counts
- ULTIMATE-deblending I. A 50-band Ultraviolet to Mid-infrared Photometric Catalog Combining Space- and Ground-based Data in the JWST/PRIMER survey
- The uncommon intracluster medium features of the first massive clusters selected independently of their baryon content
- Reconstructing redshift distributions with photometric galaxy clustering
- Quantifying the impact of detection bias from blended galaxies on cosmic shear surveys
- Dusty Clump Survival in Supernova Ejecta: Dust-Mediated Growth vs. Crushing by the Reverse Shock
- Lensing Without Borders. I. A Blind Comparison of the Amplitude of Galaxy-Galaxy Lensing Between Independent Imaging Surveys
- Gas thermodynamics meets galaxy kinematics: Joint mass measurements for eROSITA galaxy clusters
- AMICO galaxy clusters in KiDS-1000: Splashback radius from weak lensing and cluster-galaxy correlation function
- Euclid: Improving redshift distribution reconstruction using a deep-to-wide transfer function
- GalSBI-SPS: a stellar population synthesis-based galaxy population model for cosmology and galaxy evolution applications