The CAMELS project: Cosmology and Astrophysics with MachinE Learning Simulations

Abstract

We present the Cosmology and Astrophysics with MachinE Learning Simulations --CAMELS-- project. CAMELS is a suite of 4,233 cosmological simulations of $(25h^{-1}\mathrm{Mpc}{)}^3$ volume each: 2,184 state-of-the-art (magneto-)hydrodynamic simulations run with the AREPO and GIZMO codes, employing the same baryonic subgrid physics as the IllustrisTNG and SIMBA simulations, and 2,049 N-body simulations. The goal of the CAMELS project is to provide theory predictions for different observables as a function of cosmology and astrophysics, and it is the largest suite of cosmological (magneto-)hydrodynamic simulations designed to train machine learning algorithms. CAMELS contains thousands of different cosmological and astrophysical models by way of varying ${\Omega }_m$, ${\sigma }_8$, and four parameters controlling stellar and AGN feedback, following the evolution of more than 100 billion particles and fluid elements over a combined volume of $(400h^{-1}\mathrm{Mpc}{)}^3$. We describe the simulations in detail and characterize the large range of conditions represented in terms of the matter power spectrum, cosmic star formation rate density, galaxy stellar mass function, halo baryon fractions, and several galaxy scaling relations. We show that the IllustrisTNG and SIMBA suites produce roughly similar distributions of galaxy properties over the full parameter space but significantly different halo baryon fractions and baryonic effects on the matter power spectrum. This emphasizes the need for marginalizing over baryonic effects to extract the maximum amount of information from cosmological surveys. We illustrate the unique potential of CAMELS using several machine learning applications, including non-linear interpolation, parameter estimation, symbolic regression, data generation with Generative Adversarial Networks (GANs), dimensionality reduction, and anomaly detection.

arXiv comment34 pages, 19 figures, Matches published version. CAMELS webpage at https://www.camel-simulations.org

References in corpus (97)

• Deep Learning in Neural Networks: An Overview
• The cosmological simulation code GADGET-2
• The Global Schmidt Law in Star Forming Galaxies
• The EAGLE project: Simulating the evolution and assembly of galaxies and their environments
• Observational Evidence of AGN Feedback
• Populating a cluster of galaxies - I. Results at z=0
• E pur si muove: Galiliean-invariant cosmological hydrodynamical simulations on a moving mesh
• Statistical Properties of X-ray Clusters: Analytic and Numerical Comparisons
• Cosmological SPH simulations: A hybrid multi-phase model for star formation
• Modeling feedback from stars and black holes in galaxy mergers
• Simulating Galaxy Formation with the IllustrisTNG Model
• First results from the IllustrisTNG simulations: the stellar mass content of groups and clusters of galaxies
• Galaxies on FIRE (Feedback In Realistic Environments): Stellar Feedback Explains Cosmologically Inefficient Star Formation
• Physical Models of Galaxy Formation in a Cosmological Framework
• The Completed SDSS-IV extended Baryon Oscillation Spectroscopic Survey: Cosmological Implications from two Decades of Spectroscopic Surveys at the Apache Point observatory
• Properties of galaxies reproduced by a hydrodynamic simulation
• Simulating galaxy formation with black hole driven thermal and kinetic feedback
• The Co-Evolution of Galaxies and Supermassive Black Holes: Insights from Surveys of the Contemporary Universe
• Radiative transfer in a clumpy universe: IV. New synthesis models of the cosmic UV/X-ray background
• GIZMO: A New Class of Accurate, Mesh-Free Hydrodynamic Simulation Methods
• FIRE-2 Simulations: Physics versus Numerics in Galaxy Formation
• Simba: Cosmological Simulations with Black Hole Growth and Feedback
• Cosmological Simulations with TreeSPH
• A model for cosmological simulations of galaxy formation physics
• First Results from the TNG50 Simulation: The evolution of stellar and gaseous disks across cosmic time
• Ahf: Amiga's Halo Finder
• A unified model for AGN feedback in cosmological simulations of structure formation
• Dancing in the dark: galactic properties trace spin swings along the cosmic web
• A New Calculation of the Ionizing Background Spectrum and the Effects of HeII Reionization
• Chemical enrichment in cosmological, smoothed particle hydrodynamics simulations
• Gusty, gaseous flows of FIRE: galactic winds in cosmological simulations with explicit stellar feedback
• On the evolution of the HI column density distribution in cosmological simulations
• The BAHAMAS project: Calibrated hydrodynamical simulations for large-scale structure cosmology
• Feedback and Recycled Wind Accretion: Assembling the z=0 Galaxy Mass Function
• The effects of galaxy formation on the matter power spectrum: A challenge for precision cosmology
• The Cosmological Analysis of the SDSS/BOSS data from the Effective Field Theory of Large-Scale Structure
• The Arepo public code release
• Cosmological Parameters from the BOSS Galaxy Power Spectrum
• The Cosmic Baryon Cycle and Galaxy Mass Assembly in the FIRE Simulations
• Grackle: a Chemistry and Cooling Library for Astrophysics
• The Quijote simulations
• MUFASA: Galaxy Formation Simulations With Meshless Hydrodynamics
• Milky Way Satellite Census. III. Constraints on Dark Matter Properties from Observations of Milky Way Satellite Galaxies
• A physical model for cosmological simulations of galaxy formation: multi-epoch validation
• A Comparison of Methods for Determining the Molecular Content of Model Galaxies
• An Analytic Model of Angular Momentum Transport by Gravitational Torques: From Galaxies to Massive Black Holes
• Black Holes on FIRE: Stellar Feedback Limits Early Feeding of Galactic Nuclei
• Gravitational torque-driven black hole growth and feedback in cosmological simulations
• Learning to Predict the Cosmological Structure Formation
• Combining Full-Shape and BAO Analyses of Galaxy Power Spectra: A 1.6% CMB-independent constraint on H0
• Radiative and Momentum Based Mechanical AGN Feedback in a 3-Dimensional Galaxy Evolution Code
• The BlueTides Simulation: First Galaxies and Reionization
• Super-Sample Covariance in Simulations
• Quantifying baryon effects on the matter power spectrum and the weak lensing shear correlation
• Initial conditions to cosmological N-body simulations, or how to run an ensemble of simulations
• The Dust-to-Gas and Dust-to-Metals Ratio in Galaxies from z=0-6
• Modelling baryonic feedback for survey cosmology
• Supermassive black holes in cosmological simulations I: M_BH-M_star relation and black hole mass function
• Exploring the effects of galaxy formation on matter clustering through a library of simulation power spectra
• Cosmological Parameters and Neutrino Masses from the Final Planck and Full-Shape BOSS Data
• The Effect of Thermal Neutrino Motion on the Non-linear Cosmological Matter Power Spectrum
• Cosmological constraints with deep learning from KiDS-450 weak lensing maps
• Cosmic evolution of stellar quenching by AGN feedback: clues from the Horizon-AGN simulation
• Constraining $M_{\nu }$ with the Bispectrum I: Breaking Parameter Degeneracies
• The impact of stellar feedback on the structure, size and morphology of galaxies in Milky Way size dark matter haloes
• Non-Gaussian information from weak lensing data via deep learning
• Likelihood-free inference with neural compression of DES SV weak lensing map statistics
• Nuisance hardened data compression for fast likelihood-free inference
• The promising future of a robust cosmological neutrino mass measurement
• Fisher for complements: Extracting cosmology and neutrino mass from the counts-in-cells PDF
• Galaxy Power Spectrum Multipoles Covariance in Perturbation Theory
• The Diversity and Variability of Star Formation Histories in Models of Galaxy Evolution
• Nearest Neighbor distributions: new statistical measures for cosmological clustering
• Using the Marked Power Spectrum to Detect the Signature of Neutrinos in Large-Scale Structure
• The Impact of Galactic Feedback on the Circumgalactic Medium
• Weak lensing cosmology with convolutional neural networks on noisy data
• New Interpretable Statistics for Large Scale Structure Analysis and Generation
• Black Hole -- Galaxy Correlations in Simba
• The Horizon Run 5 Cosmological Hydrodynamic Simulation: Probing Galaxy Formation from Kilo- to Giga-parsec Scales
• An X-ray/SDSS sample (I): multi-phase outflow incidence and dependence on AGN luminosity
• Constraining the astrophysics and cosmology from 21cm tomography using deep learning with the SKA
• Super-resolution emulator of cosmological simulations using deep physical models
• The DIANOGA simulations of galaxy clusters: characterizing star formation in proto-clusters
• Galaxy bias and primordial non-Gaussianity: insights from galaxy formation simulations with IllustrisTNG
• Primordial non-Gaussianity without tails -- how to measure fNL with the bulk of the density PDF
• The impact of quenching on galaxy profiles in the Simba simulation
• Matter trispectrum: theoretical modelling and comparison to N-body simulations
• Galaxy power-spectrum responses and redshift-space super-sample effect
• Interpreting deep learning models for weak lensing
• Cosmological baryon transfer in the SIMBA simulations
• Separate Universe Simulations with IllustrisTNG: baryonic effects on power spectrum responses and higher-order statistics
• A Hybrid Deep Learning Approach to Cosmological Constraints From Galaxy Redshift Surveys
• Jet Feedback and the Photon Underproduction Crisis in Simba
• Removing Astrophysics in 21 cm maps with Neural Networks
• Stellar Dynamics and Star-Formation Histories of z $\sim$ 1 Radio-loud Galaxies
• Teaching neural networks to generate Fast Sunyaev Zel'dovich Maps
• On the reliability of initial conditions for dissipationless cosmological simulations

Cited by in corpus (169)

• The FLAMINGO project: cosmological hydrodynamical simulations for large-scale structure and galaxy cluster surveys
• Large-scale dark matter simulations
• Hydrodynamical simulations of the galaxy population: enduring successes and outstanding challenges
• Towards an Optimal Estimation of Cosmological Parameters with the Wavelet Scattering Transform
• The DAWES review 10: The impact of deep learning for the analysis of galaxy surveys
• Detecting Neutrino Mass by Combining Matter Clustering, Halos, and Voids
• The MillenniumTNG Project: High-precision predictions for matter clustering and halo statistics
• The CAMELS Multifield Dataset: Learning the Universe's Fundamental Parameters with Artificial Intelligence
• Five-percent measurements of the growth rate from simulation-based modelling of redshift-space clustering in BOSS LOWZ
• The CAMELS project: public data release
• Going Beyond the Galaxy Power Spectrum: an Analysis of BOSS Data with Wavelet Scattering Transforms
• Learning cosmology and clustering with cosmic graphs
• Robust Field-level Likelihood-free Inference with Galaxies
• Cosmology with one galaxy?
• Cross-correlation of DES Y3 lensing and ACT/$Planck$ thermal Sunyaev Zel'dovich Effect II: Modeling and constraints on halo pressure profiles
• $\text{Mangrove}$: Learning Galaxy Properties from Merger Trees
• The Bacco Simulation Project: Bacco Hybrid Lagrangian Bias Expansion Model in Redshift Space
• AI-assisted super-resolution cosmological simulations II: Halo substructures, velocities and higher order statistics
• Offset between X-ray and optical centers in clusters of galaxies: connecting eROSITA data and simulations
• CosmoGridV1: a simulated $w$CDM theory prediction for map-level cosmological inference
• Machine Learning for Observational Cosmology
• Precise Cosmological Constraints from BOSS Galaxy Clustering with a Simulation-Based Emulator of the Wavelet Scattering Transform
• Finding universal relations in subhalo properties with artificial intelligence
• Gravitational imaging through a triple source plane lens: revisiting the $\Lambda$CDM-defying dark subhalo in SDSSJ0946+1006
• Breaking baryon-cosmology degeneracy with the electron density power spectrum
• Euclid preparation: XIII. Forecasts for galaxy morphology with the Euclid Survey using Deep Generative Models
• Inferring the impact of feedback on the matter distribution using the Sunyaev Zel'dovich effect: Insights from CAMELS simulations and ACT+DES data
• The scatter in the galaxy-halo connection: a machine learning analysis
• Robust Simulation-Based Inference in Cosmology with Bayesian Neural Networks
• Probing the Circum-Galactic Medium with Fast Radio Bursts: Insights from the CAMELS Simulations
• Augmenting astrophysical scaling relations with machine learning: application to reducing the Sunyaev-Zeldovich flux-mass scatter
• The SZ flux-mass ($Y$-$M$) relation at low halo masses: improvements with symbolic regression and strong constraints on baryonic feedback
• From EMBER to FIRE: predicting high resolution baryon fields from dark matter simulations with Deep Learning
• HIFlow: Generating Diverse HI Maps and Inferring Cosmology while Marginalizing over Astrophysics using Normalizing Flows
• The DESI PRObabilistic Value-Added Bright Galaxy Survey (PROVABGS) Mock Challenge
• Predicting sub-millimeter flux densities from global galaxy properties
• The FLAMINGO project: Baryon effects on the matter power spectrum
• Inferring galaxy dark halo properties from visible matter with Machine Learning
• Stochastic Modelling of Star Formation Histories III. Constraints from Physically-Motivated Gaussian Processes
• Neural networks as optimal estimators to marginalize over baryonic effects
• Efficient long-range AGN feedback affects the low redshift Lyman-$\alpha$ forest
• Taming assembly bias for primordial non-Gaussianity
• Effects of Baryonic Feedback on the Cosmic Web
• Calibrating cosmological simulations with implicit likelihood inference using galaxy growth observables
• The Three Hundred project: A Machine Learning method to infer clusters of galaxies mass radial profiles from mock Sunyaev-Zel'dovich maps
• DSPS: Differentiable Stellar Population Synthesis
• Quantifying Baryonic Feedback on Warm-Hot Circumgalactic Medium in CAMELS Simulations
• Fast and realistic large-scale structure from machine-learning-augmented random field simulations
• Data Compression and Inference in Cosmology with Self-Supervised Machine Learning
• Mapping gas around massive galaxies: cross-correlation of DES Y3 galaxies and Compton-$y$-maps from SPT and Planck
• The Impacts of Modeling Choices on the Inference of the Circumgalactic Medium Properties from Sunyaev-Zeldovich Observations
• Machine Learning and Cosmology
• Percent-level constraints on baryonic feedback with spectral distortion measurements
• On the choice of the most suitable indicator for the assembly state of dark matter haloes through cosmic time
• X-raying CAMELS: Constraining Baryonic Feedback in the Circum-Galactic Medium with the CAMELS simulations and eRASS X-ray Observations
• Modelling the galaxy-halo connection with semi-recurrent neural networks
• ERGO-ML: Towards a robust machine learning model for inferring the fraction of accreted stars in galaxies from integral-field spectroscopic maps
• The impact of baryons on the internal structure of dark matter haloes from dwarf galaxies to superclusters in the redshift range 0<z<7
• Comparing weak lensing peak counts in baryonic correction models to hydrodynamical simulations
• Robust Neural Network-Enhanced Estimation of Local Primordial Non-Gaussianity
• Baryonic Imprints on DM Halos: The concentration-mass relation in the CAMELS simulations
• Forecasting the power of Higher Order Weak Lensing Statistics with automatically differentiable simulations
• Emulating Sunyaev-Zeldovich Images of Galaxy Clusters using Auto-Encoders
• Bayesian Control Variates for optimal covariance estimation with pairs of simulations and surrogates
• Not Hydro: Using Neural Networks to estimate galaxy properties on a Dark-Matter-Only simulation
• High-fidelity reproduction of central galaxy joint distributions with Neural Networks
• Towards unveiling the large-scale nature of gravity with the wavelet scattering transform
• Learning the Universe: Cosmological and Astrophysical Parameter Inference with Galaxy Luminosity Functions and Colours
• CASCO: Cosmological and AStrophysical parameters from Cosmological simulations and Observations -- I. Constraining physical processes in local star-forming galaxies
• Inferring the Equation of State from Neutron Star Observables via Machine Learning
• Quantifying Observational Projection Effects with a Simulation-based hot CGM model
• Probing galaxy evolution from $z=0$ to $z\simeq 10$ through galaxy scaling relations in three L-Galaxies flavours
• A field-level emulator for modeling baryonic effects across hydrodynamic simulations
• iMaNGA: mock MaNGA galaxies based on IllustrisTNG and MaStar SSPs. -- III. Stellar metallicity drivers in MaNGA and TNG50
• Cosmology with Galaxy Cluster Properties using Machine Learning
• Boundless baryons: how diffuse gas contributes to anisotropic tSZ signal around simulated Three Hundred clusters
• An exploration of the properties of cluster profiles for the thermal and kinetic Sunyaev-Zel'dovich effects
• Diffusion-HMC: Parameter Inference with Diffusion-model-driven Hamiltonian Monte Carlo
• Optimal machine-driven acquisition of future cosmological data
• Selection of powerful radio galaxies with machine learning
• Cosmology from one galaxy in a void?
• Correlations of Dark Matter, Gas and Stellar Profiles in Dark Matter Halos
• Modern Cosmology, an Amuse-Gueule
• BullFrog: Multi-step perturbation theory as a time integrator for cosmological simulations
• How do baryonic effects on the cosmic matter distribution vary with scale and local density environment?
• Field-level simulation-based inference with galaxy catalogs: the impact of systematic effects
• ARTEMIS emulator: exploring the effect of cosmology and galaxy formation physics on Milky Way-mass haloes and their satellites
• Perturbation Theory Remixed: Improved Nonlinearity Modeling beyond Standard Perturbation Theory
• Quantifying the intrinsic variability due to randomness of the Auriga galaxy formation model
• Modeling Post-Reionization HI Distributions in Fuzzy Dark Matter Cosmologies Using Conditional Normalizing Flows
• $\text{slick}$: Modeling a Universe of Molecular Line Luminosities in Hydrodynamical Simulations
• A rising tide: Intrinsic alignments since the turn of the millennium
• Inpainting hydrodynamical maps with deep learning
• The Three Hundred Project: Mapping The Matter Distribution in Galaxy Clusters Via Deep Learning from Multiview Simulated Observations
• MF-Box: Multi-fidelity and multi-scale emulation for the matter power spectrum
• Map-level baryonification: unified treatment of weak lensing two-point and higher-order statistics
• Discovering Nuclear Models from Symbolic Machine Learning
• Psi-GAN: A power-spectrum-informed generative adversarial network for the emulation of large-scale structure maps across cosmologies and redshifts
• Evaluating the feasibility of interpretable machine learning for globular cluster detection
• Perturbation-theory informed integrators for cosmological simulations
• Particle clustering in turbulence: Prediction of spatial and statistical properties with deep learning
• The cosmological analysis of X-ray cluster surveys: VI. Inference based on analytically simulated observable diagrams
• Secondary halo bias through cosmic time II: Reconstructing halo properties using clustering information
• Baryon Pasting the Uchuu Lightcone Simulation
• Multi-Epoch Machine Learning 2: Identifying physical drivers of galaxy properties in simulations
• Cosmology from Point Clouds with Dark Matter Halos from the Quijote Simulations
• Differentiable Cosmological Hydrodynamics for Field-Level Inference and High Dimensional Parameter Constraints
• Predictive uncertainty on improved astrophysics recovery from multifield cosmology
• Constraining $f_{NL}$ using the Large-Scale Modulation of Small-Scale Statistics
• Baryonic Imprints on DM Halos: the concentration-mass relation and its dependence on halo and galaxy properties
• The Power of the Cosmic Web
• syren-baryon: Analytic emulators for the impact of baryons on the matter power spectrum
• Simulation-based inference has its own Dodelson-Schneider effect (but it knows that it does)
• Learning the Universe: Learning to Optimize Cosmic Initial Conditions with Non-Differentiable Structure Formation Models
• kSZ Pairwise Velocity Reconstruction with Machine Learning
• Anisotropic cosmology using observational datasets: exploring via machine learning approaches
• Cosmology with One Galaxy: Auto-Encoding the Galaxy Properties Manifold
• CASCO: Cosmological and AStrophysical parameters from Cosmological simulations and Observations -- II. Constraining cosmology and astrophysical processes with early- and late-type galaxies
• Field-Level Comparison and Robustness Analysis of Cosmological N-body Simulations
• Disentangling the Halo: Joint Model for Measurements of the Kinetic Sunyaev-Zeldovich Effect and Galaxy-Galaxy Lensing
• The Cosmological analysis of X-ray cluster surveys VII. Bypassing scaling relations with Lagrangian Deep Learning and Simulation-based inference
• DARKSKIES: A suite of super-sampled zoom-in simulations of galaxy clusters with self-interacting dark matter
• Strong Evidence for Cosmic Ray-Supported $\sim$L${}^{\ast }$ Galaxy Halos via X-ray \& tSZ Constraints
• The Compressed 3D Lyman-Alpha Forest Bispectrum
• Interpreting Cosmological Information from Neural Networks in the Hydrodynamic Universe
• Cosmology with Topological Deep Learning
• On the effects of parameters on galaxy properties in CAMELS and the predictability of ${\Omega }_{\mathrm{m}}$
• Mapping luminous and dark matter in the Universe
• Map Reconstruction of radio observations with Conditional Invertible Neural Networks
• An Observationally Driven Multifield Approach for Probing the Circum-Galactic Medium with Convolutional Neural Networks
• Probabilistic Lagrangian bias estimators and the cumulant bias expansion
• Stirring the cosmic pot: how black hole feedback shapes the matter power spectrum in the Fable simulations
• The Circumgalactic Medium from the CAMELS Simulations: Forecasting Constraints on Feedback Processes from Future Sunyaev-Zeldovich Observations
• Inferring halo masses with Graph Neural Networks
• Exploring the halo-galaxy connection with probabilistic approaches
• Reconstructing the mid-infrared spectra of galaxies using ultraviolet to submillimeter photometry and Deep Generative Networks
• Parameter estimation from Ly$\alpha$ forest in Fourier space using Information Maximising Neural Network
• Deriving accurate galaxy cluster masses using X-ray thermodynamic profiles and graph neural networks
• JERALD: high-fidelity dark matter, stellar mass and neutral hydrogen maps from fast N-body simulations
• First Light And Reionization Epoch Simulations (FLARES) -- XIX: Supermassive black hole mergers in the early Universe and their environmental dependence
• Insights into the dependence of galaxy properties on the environment with explainable machine learning models
• One latent to fit them all: a unified representation of baryonic feedback on matter distribution
• Galaxy 3D Shape Recovery using Mixture Density Network
• Modeling the Cosmological Lyman-$\alpha$ Forest at the Field Level
• Towards an optimal extraction of cosmological parameters from galaxy cluster surveys using convolutional neural networks
• Foreground removal in HI 21 cm intensity mapping under frequency-dependent beam distortions
• Galaxy assembly bias in cosmological hydrodynamical simulations: a comparison between SIMBA and IllustrisTNG
• Cosmological gravity on all scales V: MCMC forecasts combining large scale structure and CMB lensing for binned phenomenological modified gravity
• A New Method to Simulate Dark Matter-Baryon Interactions and Application to an Isolated Disk Galaxy
• Mainly on the Plane: Observing the Extended, Ionized Disks of Milky Way Analogs in IllustrisTNG
• The Baryon Budget of Galaxies across the First Billion Years
• Structure Formation under Inelastic Two-Component Dark Matter: Halo Statistics and Matter Power Spectra in the High-$z$ Universe
• Can AI Dream of Unseen Galaxies? Conditional Diffusion Model for Galaxy Morphology Augmentation
• Large Language Models -- the Future of Fundamental Physics?
• Fast Baryonic Field Painting for Sunyaev-Zel'dovich Analyses: Transfer Function vs. Hybrid Effective Field Theory
• Cosmic Dynamics in Einstein-Cartan Theory: Analysing Hubble Tension through Curvature and Torsion field
• Interpretability of deep-learning methods applied to large-scale structure surveys
• Cosmological constraints from the density gradient weighted correlation function
• The halo finding problem revisited: a deep revision of the ASOHF code
• From Redshift to Real Space: Combining Linear Theory With Neural Networks
• Variational autoencoder for generating realistic $N$-body simulations for dark matter halos
• Predicting large scale cosmological structure evolution with generative adversarial network-based autoencoders
• Deep Learning generated observations of galaxy clusters from dark-matter-only simulations
• Predicting Halo Formation Time Using Machine Learning
• Sahyadri: A simulation suite for the cosmology dependence of the Cosmic Web
• Mimicking the halo-galaxy connection using machine learning
• Replicating weak-lensing summary-statistic covariances with normalizing flows
• Spatially resolved stellar-to-total dynamical mass relation: Radial variations, gradients and profiles of galaxy stellar populations
• Dark Energy Survey Year 3 results: Cosmology with peaks using an emulator approach