CARPool: fast, accurate computation of large-scale structure statistics by pairing costly and cheap cosmological simulations

Abstract

To exploit the power of next-generation large-scale structure surveys, ensembles of numerical simulations are necessary to give accurate theoretical predictions of the statistics of observables. High-fidelity simulations come at a towering computational cost. Therefore, approximate but fast simulations, surrogates, are widely used to gain speed at the price of introducing model error. We propose a general method that exploits the correlation between simulations and surrogates to compute fast, reduced-variance statistics of large-scale structure observables without model error at the cost of only a few simulations. We call this approach Convergence Acceleration by Regression and Pooling (CARPool). In numerical experiments with intentionally minimal tuning, we apply CARPool to a handful of GADGET-III $N$-body simulations paired with surrogates computed using COmoving Lagrangian Acceleration (COLA). We find $\sim 100$-fold variance reduction even in the non-linear regime, up to $k_{\max }\approx 1.2$ $h\mathrm{Mp}{\mathrm{c}}^{-1}$ for the matter power spectrum. CARPool realises similar improvements for the matter bispectrum. In the nearly linear regime CARPool attains far larger sample variance reductions. By comparing to the 15,000 simulations from the Quijote suite, we verify that the CARPool estimates are unbiased, as guaranteed by construction, even though the surrogate misses the simulation truth by up to $60\%$ at high $k$. Furthermore, even with a fully configuration-space statistic like the non-linear matter density probability density function, CARPool achieves unbiased variance reduction factors of up to $\sim 10$, without any further tuning. Conversely, CARPool can be used to remove model error from ensembles of fast surrogates by combining them with a few high-accuracy simulations.

arXiv comment18 pages, 18 figures. v2: Improved and published version; references and discussions added, typos fixed, slight visual modifications of some figures

References in corpus (64)

• The cosmological simulation code GADGET-2
• Efficient Computation of CMB anisotropies in closed FRW models
• LSST: from Science Drivers to Reference Design and Anticipated Data Products
• Large-Scale Structure of the Universe and Cosmological Perturbation Theory
• The DESI Experiment Part I: Science,Targeting, and Survey Design
• The Hyper Suprime-Cam SSP Survey: Overview and Survey Design
• Wide-Field InfrarRed Survey Telescope-Astrophysics Focused Telescope Assets WFIRST-AFTA 2015 Report
• LSST Science Book, Version 2.0
• Euclid Definition Study Report
• Why your model parameter confidences might be too optimistic -- unbiased estimation of the inverse covariance matrix
• Large-Scale Galaxy Bias
• The Effective Field Theory of Cosmological Large Scale Structures
• Renormalized Cosmological Perturbation Theory
• Resumming Cosmological Perturbations via the Lagrangian Picture: One-loop Results in Real Space and in Redshift Space
• Solving Large Scale Structure in Ten Easy Steps with COLA
• The Quijote simulations
• The Coyote Universe I: Precision Determination of the Nonlinear Matter Power Spectrum
• Cosmology with Phase 1 of the Square Kilometre Array; Red Book 2018: Technical specifications and performance forecasts
• The Uchuu Simulations: Data Release 1 and Dark Matter Halo Concentrations
• Second Order Power Spectrum and Nonlinear Evolution at High Redshift
• The DESI Experiment Part II: Instrument Design
• HACC: Simulating Sky Surveys on State-of-the-Art Supercomputing Architectures
• Learning to Predict the Cosmological Structure Formation
• FastPM: a new scheme for fast simulations of dark matter and halos
• The BACCO Simulation Project: Exploiting the full power of large-scale structure for cosmology
• EZmocks: extending the Zel'dovich approximation to generate mock galaxy catalogues with accurate clustering statistics
• Cosmological $N$-body simulations with suppressed variance
• Cosmology with the SPHEREX All-Sky Spectral Survey
• The MICE Grand Challenge Lightcone Simulation II: Halo and Galaxy catalogues
• A Lagrangian effective field theory
• Prime Focus Spectrograph (PFS) for the Subaru Telescope: Overview, recent progress, and future perspectives
• The Aemulus Project I: Numerical Simulations for Precision Cosmology
• Constraining $M_{\nu }$ with the Bispectrum I: Breaking Parameter Degeneracies
• An accurate tool for the fast generation of dark matter halo catalogs
• The Aemulus Project III: Emulation of the Galaxy Correlation Function
• The Aemulus Project II: Emulating the Halo Mass Function
• GreeM : Massively Parallel TreePM Code for Large Cosmological N-body Simulations
• A Generalized Approximate Control Variate Framework for Multifidelity Uncertainty Quantification
• The Abacus Cosmos: A Suite of Cosmological N-body Simulations
• The Outer Rim Simulation: A Path to Many-Core Supercomputers
• ICE-COLA: Towards fast and accurate synthetic galaxy catalogues optimizing a quasi $N$-body method
• U-Net: Convolutional Networks for Biomedical Image Segmentation
• Comparing approximate methods for mock catalogues and covariance matrices II: Power spectrum multipoles
• Cosmological performance of SKA HI galaxy surveys
• Dark Sky Simulations: Early Data Release
• Comparing approximate methods for mock catalogues and covariance matrices I: correlation function
• Comparing approximate methods for mock catalogues and covariance matrices III: Bispectrum
• The Mildly Non-Linear Regime of Structure Formation
• Euclid preparation: VII. Forecast validation for Euclid cosmological probes
• Super-resolution emulator of cosmological simulations using deep physical models
• Statistical properties of paired fixed fields
• Inverted initial conditions: exploring the growth of cosmic structure and voids
• PINOCCHIO and the hierarchical build-up of dark matter haloes
• Matter Power Spectrum Covariance Matrix from the DEUS-PUR ΛCDM simulations: Mass Resolution and non-Gaussian Errors
• 2HOT: An Improved Parallel Hashed Oct-Tree N-Body Algorithm for Cosmological Simulation
• Cosmic variance limited Baryon Acoustic Oscillations from the DEUS-FUR $\Lambda$CDM simulation
• Precision modelling of the matter power spectrum in a Planck-like Universe
• One-point remapping of Lagrangian perturbation theory in the mildly non-linear regime of cosmic structure formation
• sCOLA: The N-body COLA Method Extended to the Spatial Domain
• Suppressing cosmic variance with paired-and-fixed cosmological simulations: average properties and covariances of dark matter clustering statistics
• Biased Tracers in Redshift Space in the EFT of Large-Scale Structure
• The Aemulus Project IV: Emulating Halo Bias
• Reducing the Variance of Redshift Space Distortion Measurements from Mock Galaxy Catalogues with Different Lines of Sight
• Perfectly parallel cosmological simulations using spatial comoving Lagrangian acceleration

Cited by in corpus (36)

• The cosmology dependence of galaxy clustering and lensing from a hybrid $N$-body-perturbation theory model
• Detecting Neutrino Mass by Combining Matter Clustering, Halos, and Voids
• The GIGANTES dataset: precision cosmology from voids in the machine learning era
• DESI 2024 V: Full-Shape Galaxy Clustering from Galaxies and Quasars
• An optimal nonlinear method for simulating relic neutrinos
• SPT-3G D1: CMB temperature and polarization power spectra and cosmology from 2019 and 2020 observations of the SPT-3G Main field
• Fast and Credible Likelihood-Free Cosmology with Truncated Marginal Neural Ratio Estimation
• Baryon Acoustic Oscillation Theory and Modelling Systematics for the DESI 2024 results
• Aemulus $\nu$: Precise Predictions for Matter and Biased Tracer Power Spectra in the Presence of Neutrinos
• Accurate predictions from small boxes: variance suppression via the Zel'dovich approximation
• CARPool Covariance: Fast, unbiased covariance estimation for large-scale structure observables
• Precision Redshift-Space Galaxy Power Spectra using Zel'dovich Control Variates
• The DESI $N$-body Simulation Project -- II. Suppressing sample variance with fast simulations
• Quijote-PNG: Quasi-maximum likelihood estimation of Primordial Non-Gaussianity in the non-linear dark matter density field
• Cosmological Vlasov-Poisson equations for dark matter: Recent developments and connections to selected plasma problems
• Euclid preparation. XLI. Galaxy power spectrum modelling in real space
• PNG-UNITsims: Halo clustering response to primordial non-Gaussianities as a function of mass
• Ultra-large-scale approximations and galaxy clustering: debiasing constraints on cosmological parameters
• Fast and realistic large-scale structure from machine-learning-augmented random field simulations
• Mitigating the noise of DESI mocks using analytic control variates
• Bayesian Control Variates for optimal covariance estimation with pairs of simulations and surrogates
• $\text{PineTree}$: A generative, fast, and differentiable halo model for wide-field galaxy surveys
• Quijote-PNG: Quasi-maximum likelihood estimation of Primordial Non-Gaussianity in the non-linear halo density field
• Validating galaxy clustering models with Fixed & Paired and Matched-ICs simulations: application to Primordial Non-Gaussianities
• The 2-point correlation function covariance with fewer mocks
• BullFrog: Multi-step perturbation theory as a time integrator for cosmological simulations
• Renormalization group and UV completion of cosmological perturbations: Gravitational collapse as a critical phenomenon
• MF-Box: Multi-fidelity and multi-scale emulation for the matter power spectrum
• COmoving Computer Acceleration (COCA): $N$-body simulations in an emulated frame of reference
• Fast and accurate collapse-time predictions for collisionless matter
• Variance Reduction via Simultaneous Importance Sampling and Control Variates Techniques Using Vegas
• Improving the Accuracy of Halo Mass Based Statistics For Fast Approximate N-body Simulations
• Super sample covariance and the volume scaling of galaxy survey covariance matrices
• Simulation budgeting for hybrid effective field theories
• The Parameter-Level Performance of Covariance Matrix Conditioning in Cosmic Microwave Background Data Analyses
• Improving cosmological covariance matrices with machine learning