ANNz2 - photometric redshift and probability distribution function estimation using machine learning

Abstract

We present ANNz2, a new implementation of the public software for photometric redshift (photo-z) estimation of Collister and Lahav (2004), which now includes generation of full probability distribution functions (PDFs). ANNz2 utilizes multiple machine learning methods, such as artificial neural networks and boosted decision/regression trees. The objective of the algorithm is to optimize the performance of the photo-z estimation, to properly derive the associated uncertainties, and to produce both single-value solutions and PDFs. In addition, estimators are made available, which mitigate possible problems of non-representative or incomplete spectroscopic training samples. ANNz2 has already been used as part of the first weak lensing analysis of the Dark Energy Survey, and is included in the experiment's first public data release. Here we illustrate the functionality of the code using data from the tenth data release of the Sloan Digital Sky Survey and the Baryon Oscillation Spectroscopic Survey. The code is available for download at https://github.com/IftachSadeh/ANNZ .

V2 - Completely revamped version of the paper

References in corpus (9)

• The Zurich Extragalactic Bayesian Redshift Analyzer (ZEBRA) and its first application: COSMOS
• Estimating the Redshift Distribution of Faint Galaxy Samples
• Photometric redshift analysis in the Dark Energy Survey Science Verification data
• Cosmology from Cosmic Shear with DES Science Verification Data
• Photometric Redshifts of Galaxies in COSMOS
• Precision photometric redshift calibration for galaxy-galaxy weak lensing
• Optimal filter systems for photometric redshift estimation
• Feature importance for machine learning redshifts applied to SDSS galaxies
• Data augmentation for machine learning redshifts applied to SDSS galaxies

Cited by in corpus (129)

• First measurement of the Hubble constant from a dark standard siren using the Dark Energy Survey galaxies and the LIGO/Virgo binary-black-hole merger GW170814
• A gravitational-wave measurement of the Hubble constant following the second observing run of Advanced LIGO and Virgo
• The third data release of the Kilo-Degree Survey and associated data products
• Dark Energy Survey Year 3 Results: Photometric Data Set for Cosmology
• Dark Energy Survey Year 1 Results: Redshift distributions of the weak lensing source galaxies
• Cosmological Inference using Gravitational Wave Standard Sirens: A Mock Data Challenge
• Cosmology from Cosmic Shear with DES Science Verification Data
• Photometric redshift estimation via deep learning
• The Strong Gravitational Lens Finding Challenge
• Redshift distributions of galaxies in the DES Science Verification shear catalogue and implications for weak lensing
• Models and Simulations for the Photometric LSST Astronomical Time Series Classification Challenge (PLAsTiCC)
• Surveying the reach and maturity of machine learning and artificial intelligence in astronomy
• Weak-lensing mass calibration of redMaPPer galaxy clusters in Dark Energy Survey Science Verification data
• Photometric Redshifts for Next-Generation Surveys
• The DAWES review 10: The impact of deep learning for the analysis of galaxy surveys
• Radio Galaxy Zoo: Compact and extended radio source classification with deep learning
• Galaxy clustering in the DESI Legacy Survey and its imprint on the CMB
• Identifying Strong Lenses with Unsupervised Machine Learning using Convolutional Autoencoder
• Evaluation of probabilistic photometric redshift estimation approaches for The Rubin Observatory Legacy Survey of Space and Time (LSST)
• Dark Energy Survey Year 3 results: Cosmological constraints from galaxy clustering and galaxy-galaxy lensing using the MagLim lens sample
• Dark Energy Survey Year 3 Results: Optimizing the Lens Sample in Combined Galaxy Clustering and Galaxy-Galaxy Lensing Analysis
• METAPHOR: A machine learning based method for the probability density estimation of photometric redshifts
• Photometric redshifts for the Kilo-Degree Survey. Machine-learning analysis with artificial neural networks
• Photometric Redshifts with the LSST: Evaluating Survey Observing Strategies
• Cosmology from large scale galaxy clustering and galaxy-galaxy lensing with Dark Energy Survey Science Verification data
• The PAU Survey: Early demonstration of photometric redshift performance in the COSMOS field
• The Weak Lensing Radial Acceleration Relation: Constraining Modified Gravity and Cold Dark Matter theories with KiDS-1000
• Photometric Redshift Estimation with a Convolutional Neural Network: NetZ
• Separation of pulsar signals from noise with supervised machine learning algorithms
• Studying galaxy troughs and ridges using Weak Gravitational Lensing with the Kilo-Degree Survey
• Cross-correlation of gravitational lensing from DES Science Verification data with SPT and Planck lensing
• Bright galaxy sample in the Kilo-Degree Survey Data Release 4: selection, photometric redshifts, and physical properties
• Morpho-z: improving photometric redshifts with galaxy morphology
• Selection biases in empirical p(z) methods for weak lensing
• KiDS-1000: Combined halo-model cosmology constraints from galaxy abundance, galaxy clustering and galaxy-galaxy lensing
• 2dFLenS and KiDS: Determining source redshift distributions with cross-correlations
• A machine learning approach to galaxy properties: joint redshift-stellar mass probability distributions with Random Forest
• A new method to assign galaxy cluster membership using photometric redshifts
• Mysterious Odd Radio Circle near the Large Magellanic Cloud -- An Intergalactic Supernova Remnant?
• Cross-Correlation Redshift Calibration Without Spectroscopic Calibration Samples in DES Science Verification Data
• The MUSE Hubble Ultra Deep Field Survey: III. Testing photometric redshifts to 30th magnitude
• Improving the reliability of photometric redshift with machine learning
• Photometric redshifts with machine learning, lights and shadows on a complex data science use case
• Approximating photo-$z$ PDFs for large surveys
• Photometric redshifts and Galaxy Clusters for DES DR2, DESI DR9, and HSC-SSP PDR3 Data
• Estimating the Mass of the Local Group using Machine Learning Applied to Numerical Simulations
• KiDS-1000 Cosmology: Constraints from density split statistics
• Photometric redshifts for X-ray-selected active galactic nuclei in the eROSITA era
• DES Science Portal: Computing Photometric Redshifts
• Statistical analysis of probability density functions for photometric redshifts through the KiDS-ESO-DR3 galaxies
• GAlaxy Light profile convolutional neural NETworks (GaLNets). I. fast and accurate structural parameters for billion galaxy samples
• Radial acceleration relation of galaxies with joint kinematic and weak-lensing data
• Luminous red galaxies in the Kilo Degree Survey: selection with broad-band photometry and weak lensing measurements
• Organised Randoms: Learning and correcting for systematic galaxy clustering patterns in KiDS using self-organising maps
• Extracting Photometric Redshift from Galaxy Flux and Image Data using Neural Networks in the CSST Survey
• The PAU Survey: narrowband photometric redshifts using Gaussian processes
• Machine Learning Based Real Bogus System for HSC-SSP Moving Object Detecting Pipeline
• A Joint Analysis for Cosmology and Photometric Redshift Calculation Using Cross Correlations
• Measuring the Hubble Constant with cosmic chronometers: a machine learning approach
• The PAU Survey: Intrinsic alignments and clustering of narrow-band photometric galaxies
• Comparison of Observed Galaxy Properties with Semianalytic Model Predictions using Machine Learning
• Augmenting machine learning photometric redshifts with Gaussian mixture models
• Photometric redshift estimates using Bayesian neural networks in the CSST survey
• HI Intensity Mapping for Clustering-Based Redshift Estimation
• Preliminary results of using k-Nearest Neighbours Regression to estimate the redshift of radio selected datasets
• Bayesian photometric redshifts of blended sources
• Hybrid photometric redshifts for sources in the COSMOS and XMM-LSS fields
• PhotoRedshift-MML: a multimodal machine learning method for estimating photometric redshifts of quasars
• Galaxy morphoto-Z with neural Networks (GaZNets). I. Optimized accuracy and outlier fraction from Imaging and Photometry
• Astronomaly at scale: searching for anomalies amongst 4 million galaxies
• The Spatial Correlation of Bent-Tail Galaxies and Galaxy Clusters
• Deep Learning in Searching the Spectroscopic Redshift of Quasars
• Photometric redshifts for the Pan-STARRS1 survey
• Clustering with general photo-$z$ uncertainties: Application to Baryon Acoustic Oscillations
• Galaxy bias from galaxy-galaxy lensing in the DES Science Verification Data
• TOPz: Photometric redshifts for J-PAS
• Galaxy Tagging: photometric redshift refinement and group richness enhancement
• A DECam Search for Explosive Optical Transients Associated with IceCube Neutrinos
• An adapted filter function for density split statistics in weak lensing
• Morpho-Photometric Redshifts
• Simulation-based inference of deep fields: galaxy population model and redshift distributions
• High-fidelity reproduction of central galaxy joint distributions with Neural Networks
• Machine learning synthetic spectra for probabilistic redshift estimation: SYTH-Z
• On the cosmological performance of photometrically classified supernovae with machine learning
• PICZL: Image-based Photometric Redshifts for AGN
• Data Deluge in Astrophysics: Photometric Redshifts as a Template Use Case
• A Machine Learning Approach to Galaxy-LSS Classification I: Imprints on Halo Merger Trees
• Machine learning technique for morphological classification of galaxies from SDSS. II. The image-based morphological catalogs of galaxies at 0.02<z<0.1
• Machine learning technique for morphological classification of galaxies from the SDSS. III. Image-based inference of detailed features
• Intrinsic galaxy alignments in the KiDS-1000 bright sample: dependence on colour, luminosity, morphology, and galaxy scale
• Convolutional Neural Networks for Spectroscopic Redshift Estimation on Euclid Data
• Euclid preparation. LXVIII. Extracting physical parameters from galaxies with machine learning
• Estimating Photometric Redshifts for Galaxies from the DESI Legacy Imaging Surveys with Bayesian Neural Networks Trained by DESI EDR
• Halo shapes constrained from a pure sample of central galaxies in KiDS-1000
• Impact of photometric redshifts on the galaxy power spectrum and BAO scale in the LSST survey
• The many flavours of photometric redshifts
• The effect of emission lines on the performance of photometric redshift estimation algorithms
• Predicting bulge to total luminosity ratio of galaxies using deep learning
• Enhancing Photometric Redshift Catalogs Through Color-Space Analysis: Application to KiDS-Bright Galaxies
• Gaussian Mixture Models for Blended Photometric Redshifts
• Degradation analysis in the estimation of photometric redshifts from non-representative training sets
• Rejection criteria based on outliers in the KiDS photometric redshifts and PDF distributions derived by machine learning
• Dark Energy Survey Deep Field photometric redshift performance and training incompleteness assessment
• Machine-learning the spectral function of a hole in a quantum antiferromagnet
• Augmenting photometric redshift estimates using spectroscopic nearest neighbours
• Machine learning applications for the study of AGN physical properties using photometric observations
• Euclid: Photometric redshift calibration with self-organising maps
• The Sensitivity of the Redshift Distribution to Galaxy Demographics
• Machine learning applications in astrophysics: Photometric redshift estimation
• CLAP. I. Resolving miscalibration for deep learning-based galaxy photometric redshift estimation
• The VST ATLAS Quasar Survey I: Catalogue
• Optimizing Redshift Distribution Inference through Joint Self-Calibration and Clustering-Redshift Synergy
• The environment of QSO triplets at 1 $\lesssim$ z $\lesssim$ 1.5
• VST ATLAS Galaxy Cluster Catalogue I: cluster detection and mass calibration
• Pz Cats: Photometric redshift catalogs based on DES Y3 BAO sample
• Influence of photometric galaxies redshift distribution in BAO estimation
• Improving Photometric Redshifts by Merging Probability Density Functions from Template-Based and Machine Learning Algorithms
• Early Detection of Multiwavelength Blazar Variability
• Characterising Improvements in Photometric Redshift Probability Density Functions with Galaxy Morphology
• Estimating Cluster Masses from SDSS Multi-band Images with Transfer Learning
• Deriving Photometric Redshifts using Fuzzy Archetypes and Self-Organizing Maps. II. Comparing Sampling Techniques Using Mock Data
• DeepRed: an architecture for redshift estimation
• Searching for Dark Structures: A Comparison of Weak-lensing Convergence Maps and Lensing-weighted Galaxy Density Maps
• Machine learning analysis of Photometric data from the Dark Energy Survey
• A catalog of photometric redshift and the distribution of broad galaxy morphologies
• TOPz: Photometric redshifts using template fitting applied to the GAMA survey
• Hybrid-z: Enhancing Kilo-Degree Survey bright galaxy sample photometric redshifts with deep learning
• Photometric redshifts for the S-PLUS Survey: is machine learning up to the task?
• Photometric Redshift Estimation Using Scaled Ensemble Learning