Machine Learning for Observational Cosmology

Abstract

An array of large observational programs using ground-based and space-borne telescopes is planned in the next decade. The forthcoming wide-field sky surveys are expected to deliver a sheer volume of data exceeding an exabyte. Processing the large amount of multiplex astronomical data is technically challenging, and fully automated technologies based on machine learning and artificial intelligence are urgently needed. Maximizing scientific returns from the big data requires community-wide efforts. We summarize recent progress in machine learning applications in observational cosmology. We also address crucial issues in high-performance computing that are needed for the data processing and statistical analysis.

arXiv comment55 pages, 8 figures, accepted for publication in Reports on Progress in Physics

References in corpus (248)

• LSST: from Science Drivers to Reference Design and Anticipated Data Products
• The clustering of galaxies in the completed SDSS-III Baryon Oscillation Spectroscopic Survey: cosmological analysis of the DR12 galaxy sample
• The 2dF Galaxy Redshift Survey: Power-spectrum analysis of the final dataset and cosmological implications
• Stable clustering, the halo model and nonlinear cosmological power spectra
• Halo Models of Large Scale Structure
• The Simons Observatory: Science goals and forecasts
• Physical Models of Galaxy Formation in a Cosmological Framework
• The Completed SDSS-IV extended Baryon Oscillation Spectroscopic Survey: Cosmological Implications from two Decades of Spectroscopic Surveys at the Apache Point observatory
• UniverseMachine: The Correlation between Galaxy Growth and Dark Matter Halo Assembly from z=0-10
• Explaining Machine Learning Classifiers through Diverse Counterfactual Explanations
• Revising the Halofit Model for the Nonlinear Matter Power Spectrum
• Spectroscopic Target Selection for the Sloan Digital Sky Survey: The Luminous Red Galaxy Sample
• Unmasking Clever Hans Predictors and Assessing What Machines Really Learn
• Cosmology and fundamental physics with the Euclid satellite
• Large-Scale Galaxy Bias
• KiDS-1000 Cosmology: Cosmic shear constraints and comparison between two point statistics
• Reconstructing the linear power spectrum of cosmological mass fluctuations
• Galaxy Zoo 1 : Data Release of Morphological Classifications for nearly 900,000 galaxies
• Extragalactic Science, Cosmology and Galactic Archaeology with the Subaru Prime Focus Spectrograph (PFS)
• Galaxy Zoo 2: detailed morphological classifications for 304,122 galaxies from the Sloan Digital Sky Survey
• The Electromagnetic Counterpart of the Binary Neutron Star Merger LIGO/Virgo GW170817. I. Dark Energy Camera Discovery of the Optical Counterpart
• Cosmology from cosmic shear power spectra with Subaru Hyper Suprime-Cam first-year data
• The 6dF Galaxy Survey: z \approx 0 measurement of the growth rate and sigma_8
• An accurate halo model for fitting non-linear cosmological power spectra and baryonic feedback models
• Dark Energy Survey Year 3 Results: Cosmology from Cosmic Shear and Robustness to Modeling Uncertainty
• Weak lensing for precision cosmology
• The Zwicky Transient Facility: Surveys and Scheduler
• Third Data Release of the Hyper Suprime-Cam Subaru Strategic Program
• The Quijote simulations
• Photometric Redshifts for Hyper Suprime-Cam Subaru Strategic Program Data Release 1
• The Coyote Universe I: Precision Determination of the Nonlinear Matter Power Spectrum
• The fourth data release of the Kilo-Degree Survey: ugri imaging and nine-band optical-IR photometry over 1000 square degrees
• The Dark Energy Survey Data Release 2
• Bayesian physical reconstruction of initial conditions from large scale structure surveys
• The CAMELS project: Cosmology and Astrophysics with MachinE Learning Simulations
• The WiggleZ Dark Energy Survey: Final data release and cosmological results
• A catalog of visual-like morphologies in the 5 CANDELS fields using deep-learning
• Fast Automated Analysis of Strong Gravitational Lenses with Convolutional Neural Networks
• CFHTLenS revisited: assessing concordance with Planck including astrophysical systematics
• The Subaru FMOS galaxy redshift survey (FastSound). IV. New constraint on gravity theory from redshift space distortions at $z\sim 1.4$
• The Coyote Universe II: Cosmological Models and Precision Emulation of the Nonlinear Matter Power Spectrum
• The Coyote Universe Extended: Precision Emulation of the Matter Power Spectrum
• The Sloan Digital Sky Survey-II Supernova Survey: Search Algorithm and Follow-up Observations
• AbacusSummit: A Massive Set of High-Accuracy, High-Resolution $N$-Body Simulations
• The third data release of the Kilo-Degree Survey and associated data products
• Learning to Predict the Cosmological Structure Formation
• Dark Quest. I. Fast and Accurate Emulation of Halo Clustering Statistics and Its Application to Galaxy Clustering
• The VIMOS Public Extragalactic Redshift Survey (VIPERS): The growth of structures at $0.5<z<1.2$ from redshift-space distortions in the clustering of the PDR-2 final sample
• The BACCO Simulation Project: Exploiting the full power of large-scale structure for cosmology
• Photometric Supernova Classification With Machine Learning
• The Coyote Universe III: Simulation Suite and Precision Emulator for the Nonlinear Matter Power Spectrum
• Photometric redshifts from SDSS images using a Convolutional Neural Network
• Euclid preparation: II. The EuclidEmulator -- A tool to compute the cosmology dependence of the nonlinear matter power spectrum
• Finding Strong Gravitational Lenses in the Kilo Degree Survey with Convolutional Neural Networks
• COSMOPOWER: emulating cosmological power spectra for accelerated Bayesian inference from next-generation surveys
• Large-scale dark matter simulations
• Real-bogus classification for the Zwicky Transient Facility using deep learning
• Results from the Supernova Photometric Classification Challenge
• Euclid preparation: IX. EuclidEmulator2 -- Power spectrum emulation with massive neutrinos and self-consistent dark energy perturbations
• Cosmology Constraints from the Weak Lensing Peak Counts and the Power Spectrum in CFHTLenS
• The Effective Number Density of Galaxies for Weak Lensing Measurements in the LSST Project
• Automating Discovery and Classification of Transients and Variable Stars in the Synoptic Survey Era
• The Strong Gravitational Lens Finding Challenge
• RAPID: Early Classification of Explosive Transients using Deep Learning
• The Core-collapse rate from the Supernova Legacy Survey
• The Aemulus Project I: Numerical Simulations for Precision Cosmology
• Alert Classification for the ALeRCE Broker System: The Light Curve Classifier
• The Mira-Titan Universe: Precision Predictions for Dark Energy Surveys
• The Mira-Titan Universe II: Matter Power Spectrum Emulation
• The Aemulus Project III: Emulation of the Galaxy Correlation Function
• The Aemulus Project II: Emulating the Halo Mass Function
• A Measurement of the Rate of type-Ia Supernovae at Redshift $z\approx$ 0.1 from the First Season of the SDSS-II Supernova Survey
• Non-Gaussian information from weak lensing data via deep learning
• MassiveNuS: Cosmological Massive Neutrino Simulations
• Using Machine Learning for Discovery in Synoptic Survey Imaging
• Simulations of Baryon Acoustic Oscillations II: Covariance matrix of the matter power spectrum
• Cosmic Calibration: Constraints from the Matter Power Spectrum and the Cosmic Microwave Background
• Fast Hamiltonian sampling for large scale structure inference
• Avocado: Photometric Classification of Astronomical Transients with Gaussian Process Augmentation
• Euclid preparation: X. The Euclid photometric-redshift challenge
• LinKS: Discovering galaxy-scale strong lenses in the Kilo-Degree Survey using Convolutional Neural Networks
• Supernova cosmology: legacy and future
• Emulating Simulations of Cosmic Dawn for 21cm Power Spectrum Constraints on Cosmology, Reionization, and X-ray Heating
• CosmoGAN: creating high-fidelity weak lensing convergence maps using Generative Adversarial Networks
• Likelihood-Free Cosmological Inference with Type Ia Supernovae: Approximate Bayesian Computation for a Complete Treatment of Uncertainty
• Emulating the CFHTLenS Weak Lensing data: Cosmological Constraints from moments and Minkowski functionals
• Dependence of cosmic shear covariances on cosmology - Impact on parameter estimation
• Photometric Redshifts for Next-Generation Surveys
• Towards optimal extraction of cosmological information from nonlinear data
• Deep Recurrent Neural Networks for Supernovae Classification
• Deep learning from 21-cm tomography of the Cosmic Dawn and Reionization
• Full-shape cosmology analysis of SDSS-III BOSS galaxy power spectrum using emulator-based halo model: a $5\%$ determination of ${\sigma }_8$
• The Abacus Cosmos: A Suite of Cosmological N-body Simulations
• Cosmic Emulation: Fast Predictions for the Galaxy Power Spectrum
• Finding Strong Gravitational Lenses in the DESI DECam Legacy Survey
• AI-assisted super-resolution cosmological simulations
• Weak lensing cosmology with convolutional neural networks on noisy data
• On the assumption of Gaussianity for cosmological two-point statistics and parameter dependent covariance matrices
• Discovering New Strong Gravitational Lenses in the DESI Legacy Imaging Surveys
• All-purpose, all-sky photometric redshifts for the Legacy Imaging Surveys Data Release 8
• Bayesian non-linear large scale structure inference of the Sloan Digital Sky Survey data release 7
• The cosmology dependence of galaxy clustering and lensing from a hybrid $N$-body-perturbation theory model
• Cosmic Calibration
• SuperRAENN: A Semi-supervised Supernova Photometric Classification Pipeline Trained on Pan-STARRS1 Medium Deep Survey Supernovae
• Analysing the 21cm signal from the Epoch of Reionization with artificial neural networks
• Approximate Bayesian Computation for Forward Modeling in Cosmology
• Emulation of reionization simulations for Bayesian inference of astrophysics parameters using neural networks
• Galaxy morphological classification in deep-wide surveys via unsupervised machine learning
• Simulations and symmetries
• cosmoabc: Likelihood-free inference via Population Monte Carlo Approximate Bayesian Computation
• Cosmological constraints from noisy convergence maps through deep learning
• Fast cosmological parameter estimation using neural networks
• Unsupervised self-organised mapping: a versatile empirical tool for object selection, classification and redshift estimation in large surveys
• Cosmic Shear Cosmology Beyond 2-Point Statistics: A Combined Peak Count and Correlation Function Analysis of DES-Y1
• A Machine Learning Approach for Dynamical Mass Measurements of Galaxy Clusters
• Evaluation of probabilistic photometric redshift estimation approaches for The Rubin Observatory Legacy Survey of Space and Time (LSST)
• Constraining neutrino mass with tomographic weak lensing one-point probability distribution function and power spectrum
• Updated design of the CMB polarization experiment satellite LiteBIRD
• Painting with baryons: augmenting N-body simulations with gas using deep generative models
• Bayesian optimisation for likelihood-free cosmological inference
• Photometric redshifts for the Kilo-Degree Survey. Machine-learning analysis with artificial neural networks
• DeepCMB: Lensing Reconstruction of the Cosmic Microwave Background with Deep Neural Networks
• Machine learning for transient discovery in Pan-STARRS1 difference imaging
• How to Find More Supernovae with Less Work: Object Classification Techniques for Difference Imaging
• A new model to predict weak-lensing peak counts II. Parameter constraint strategies
• Painting galaxies into dark matter halos using machine learning
• A Framework for Telescope Schedulers: With Applications to the Large Synoptic Survey Telescope
• The CAMELS Multifield Dataset: Learning the Universe's Fundamental Parameters with Artificial Intelligence
• Super-resolution emulator of cosmological simulations using deep physical models
• Photometric Redshift Estimation with a Convolutional Neural Network: NetZ
• Deep learning dark matter map reconstructions from DES SV weak lensing data
• Supernova Photometric Classification Pipelines Trained on Spectroscopically Classified Supernovae from the Pan-STARRS1 Medium-Deep Survey
• Cosmological Forecast for non-Gaussian Statistics in large-scale weak Lensing Surveys
• Constraining neutrino mass with weak lensing Minkowski Functionals
• Constraining Neutrino Mass with the Tomographic Weak Lensing Bispectrum
• deep21: a Deep Learning Method for 21cm Foreground Removal
• Constraining neutrino mass with tomographic weak lensing peak counts
• PS1-STRM: Neural network source classification and photometric redshift catalogue for PS1 $3\pi$ DR1
• GalaxyNet: Connecting galaxies and dark matter haloes with deep neural networks and reinforcement learning in large volumes
• {\sc CosmoNet}: fast cosmological parameter estimation in non-flat models using neural networks
• Cosmic Shear Covariance Matrix in $w$CDM: Cosmology Matters
• Galaxy Morphological Classification Catalogue of the Dark Energy Survey Year 3 data with Convolutional Neural Networks
• Machine Learning and Cosmological Simulations II: Hydrodynamical Simulations
• Photometry of high-redshift blended galaxies using deep learning
• An Emulator for the Lyman-alpha Forest
• High-quality strong lens candidates in the final Kilo Degree survey footprint
• Accurate emulator for the redshift-space power spectrum of dark matter halos and its application to galaxy power spectrum
• DASH: Deep Learning for the Automated Spectral Classification of Supernovae and their Hosts
• Cosmology with galaxy-galaxy lensing on non-perturbative scales: Emulation method and application to BOSS LOWZ
• HOLISMOKES. VI. New galaxy-scale strong lens candidates from the HSC-SSP imaging survey
• Identifying Reionization Sources from 21cm Maps using Convolutional Neural Networks
• Cosmological inference from Bayesian forward modelling of deep galaxy redshift surveys
• On the road to percent accuracy II: calibration of the non-linear matter power spectrum for arbitrary cosmologies
• Matter Power Spectrum Covariance Matrix from the DEUS-PUR ΛCDM simulations: Mass Resolution and non-Gaussian Errors
• Cosmic Emulation: The Concentration-Mass Relation for wCDM Universes
• First Cosmology Results Using Type Ia Supernovae From the Dark Energy Survey: Survey Overview and Supernova Spectroscopy
• PkANN - II. A non-linear matter power spectrum interpolator developed using artificial neural networks
• Approximate Bayesian Computation in Large Scale Structure: constraining the galaxy-halo connection
• Rico: An Accurate Cosmological Recombination Code
• Measuring our universe from galaxy redshift surveys
• FlowPM: Distributed TensorFlow Implementation of the FastPM Cosmological N-body Solver
• A Deep Learning Approach for Active Anomaly Detection of Extragalactic Transients
• Full-sky Cosmic Microwave Background Foreground Cleaning Using Machine Learning
• The skewed weak lensing likelihood: why biases arise, despite data and theory being sound
• An emulator for the Lyman-$\alpha$ forest in beyond-$\Lambda$CDM cosmologies
• Deblending galaxies with Variational Autoencoders: a joint multi-band, multi-instrument approach
• Interpreting deep learning models for weak lensing
• Unbiased Cosmology Inference from Biased Tracers using the EFT Likelihood
• PkANN - I. Non-linear matter power spectrum interpolation through artificial neural networks
• Machine learning astrophysics from 21 cm lightcones: impact of network architectures and signal contamination
• Investigating Deep Learning Methods for Obtaining Photometric Redshift Estimations from Images
• Convolutional Neural Networks for Transient Candidate Vetting in Large-Scale Surveys
• A Hybrid Deep Learning Approach to Cosmological Constraints From Galaxy Redshift Surveys
• Machine-assisted Semi-Simulation Model (MSSM): Estimating Galactic Baryonic Properties from their Dark Matter using a Machine Trained on Hydrodynamic Simulations
• The effect on cosmological parameter estimation of a parameter dependent covariance matrix
• Cosmology Inference from Biased Tracers using the EFT-based Likelihood
• Deep Learning for Strong Lensing Search: Tests of the Convolutional Neural Networks and New Candidates from KiDS DR3
• $\text{Mangrove}$: Learning Galaxy Properties from Merger Trees
• Separating the EoR Signal with a Convolutional Denoising Autoencoder: A Deep-learning-based Method
• Data augmentation for machine learning redshifts applied to SDSS galaxies
• STACCATO: A Novel Solution to Supernova Photometric Classification with Biased Training Sets
• Transient-optimised real-bogus classification with Bayesian Convolutional Neural Networks -- sifting the GOTO candidate stream
• Photometric redshifts with Quasi Newton Algorithm (MLPQNA). Results in the PHAT1 contest
• Copula Cosmology: Constructing a Likelihood Function
• Exploiting Low-Dimensional Structure in Astronomical Spectra
• Noise reduction for weak lensing mass mapping: An application of generative adversarial networks to Subaru Hyper Suprime-Cam first-year data
• The EFT Likelihood for Large-Scale Structure
• Removing Astrophysics in 21 cm maps with Neural Networks
• Probing dark energy with tomographic weak-lensing aperture mass statistics
• Star-galaxy classification in the Dark Energy Survey Y1 dataset
• AI-assisted super-resolution cosmological simulations II: Halo substructures, velocities and higher order statistics
• Deep learning approach for identification of HII regions during reionization in 21-cm observations
• Anomaly detection in Hyper Suprime-Cam galaxy images with generative adversarial networks
• Cosmological parameter estimation via iterative emulation of likelihoods
• Kernel-Based Emulator for the 3D Matter Power Spectrum from CLASS
• Exploring the spectroscopic diversity of type Ia supernovae with DRACULA: a machine learning approach
• Finding universal relations in subhalo properties with artificial intelligence
• The combined and respective roles of imaging and stellar kinematics in identifying galaxy merger remnants
• Sigma-Eight at the Percent Level: The EFT Likelihood in Real Space
• Machine-learning Selection of Optical Transients in Subaru/Hyper Suprime-Cam Survey
• The Hyper Suprime-Cam SSP Transient Survey in COSMOS: Overview
• A First Look at creating mock catalogs with machine learning techniques
• Weak lensing statistics from the Coyote Universe
• Automated novelty detection in the WISE survey with one-class support vector machines
• Multi-Fidelity Emulation for the Matter Power Spectrum using Gaussian Processes
• NECOLA: Towards a Universal Field-level Cosmological Emulator
• Exploring the posterior surface of the large scale structure reconstruction
• Photometric redshifts with machine learning, lights and shadows on a complex data science use case
• Benchmarking and Scalability of Machine Learning Methods for Photometric Redshift Estimation
• $\text{matryoshka}$: Halo Model Emulator for the Galaxy Power Spectrum
• Parameter Inference for Weak Lensing using Gaussian Processes and MOPED
• Limits on the ultra-bright Fast Radio Burst population from the CHIME Pathfinder
• Recovering the CMB Signal with Machine Learning
• Intelligent Design: On the Emulation of Cosmological Simulations
• Vetting the optical transient candidates detected by the GWAC network using convolutional neural networks
• HInet: Generating neutral hydrogen from dark matter with neural networks
• The Photometric LSST Astronomical Time-series Classification Challenge (PLAsTiCC): Selection of a performance metric for classification probabilities balancing diverse science goals
• Generating Synthetic Cosmological Data with GalSampler
• Extracting cosmological parameters from N-body simulations using machine learning techniques
• MillimeterDL: Deep Learning Simulations of the Microwave Sky
• Predicting the Neutral Hydrogen Content of Galaxies From Optical Data Using Machine Learning
• Improved supervised learning methods for EoR parameters reconstruction
• Inferring Astrophysics and Dark Matter Properties from 21cm Tomography using Deep Learning
• Photometric Classification of Early-Time Supernova Lightcurves with SCONE
• Field-level inference of galaxy intrinsic alignment from the SDSS-III BOSS survey
• Deep learning for intensity mapping observations: Component extraction
• A unified framework for 21cm tomography sample generation and parameter inference with Progressively Growing GANs
• Inpainting Galactic Foreground Intensity and Polarization maps using Convolutional Neural Network
• Extracting the 21-cm Power Spectrum and the reionization parameters from mock datasets using Artificial Neural Networks
• CFHTLenS: A Gaussian likelihood is a sufficient approximation for a cosmological analysis of third-order cosmic shear statistics
• MeerCRAB: MeerLICHT Classification of Real and Bogus Transients using Deep Learning
• Learning from the machine: interpreting machine learning algorithms for point- and extended- source classification
• The EFT Likelihood for Large-Scale Structure in Redshift Space
• Using Artificial Neural Networks to extract the 21-cm Global Signal from the EDGES data
• BAO scale inference from biased tracers using the EFT likelihood
• Cosmological forecasts with the clustering of weak lensing peaks
• Effect of model-dependent covariance matrix for studying Baryon Acoustic Oscillations
• Hybrid analytic and machine-learned baryonic property insertion into galactic dark matter haloes
• Percent-level constraints on baryonic feedback with spectral distortion measurements
• Inpainting CMB maps using Partial Convolutional Neural Networks
• Photometric classification of HSC transients using machine learning
• Searching for Anomalies in the ZTF Catalog of Periodic Variable Stars
• Deep learning reconstruction of three-dimensional galaxy distributions with intensity mapping observations
• Predicting 21cm-line map from Lyman $\alpha$ emitter distribution with Generative Adversarial Networks
• Analysing the Epoch of Reionization with three-point correlation functions and machine learning techniques
• A sparse regression approach to modeling the relation between galaxy stellar masses and their host halos
• Deep-learning Real/Bogus classification for the Tomo-e Gozen transient survey
• Using Artificial Intelligence and real galaxy images to constrain parameters in galaxy formation simulations

Cited by in corpus (19)

• The CosmoVerse White Paper: Addressing observational tensions in cosmology with systematics and fundamental physics
• CSST Cosmological Emulator I: Matter Power Spectrum Emulation with one percent accuracy to k = 10 h/Mpc
• Optimal, fast, and robust inference of reionization-era cosmology with the 21cmPIE-INN
• Classifying binary black holes from Population III stars with the Einstein Telescope: A machine-learning approach
• Distinguishing Coupled Dark Energy Models with Neural Networks
• CSST Cosmological Emulator II: Generalized Accurate Halo Mass Function Emulation
• SKATR: A Self-Supervised Summary Transformer for SKA
• Deep learning inference with the Event Horizon Telescope II. The Zingularity framework for Bayesian artificial neural networks
• Ly$\alpha$NNA: A Deep Learning Field-level Inference Machine for the Lyman-$\alpha$ Forest
• Mixing Bispectrum Multipoles Under Geometric Distortions
• Anisotropic cosmology using observational datasets: exploring via machine learning approaches
• Datacube segmentation via Deep Spectral Clustering
• CosmoGLINT: Cosmological Generative Model for Line Intensity Mapping with Transformer
• Neural Networks for cosmological model selection and feature importance using Cosmic Microwave Background data
• A machine learning approach to estimate mid-infrared fluxes from WISE data
• From Dark Matter Minihalos to Large-Scale Radiative Feedback: A Self-Consistent 3D Simulation of the First Stars and Galaxies using Neural Networks
• Virtual Realities: Is there only one advanced image display that astronomers need?
• Neural network emulator to constrain the high-$z$ IGM thermal state from Lyman-$\alpha$ forest flux auto-correlation function
• A decade of airborne electromagnetic surveying Lake Menindee (Australia) under varying water levels