Improving cosmological covariance matrices with machine learning

Abstract

Cosmological covariance matrices are fundamental for parameter inference, since they are responsible for propagating uncertainties from the data down to the model parameters. However, when data vectors are large, in order to estimate accurate and precise matrices we need huge numbers of observations, or rather costly simulations - neither of which may be viable. In this work we propose a machine learning approach to alleviate this problem in the context of the matrices used in the study of large-scale structure. With only a small amount of data (matrices built with samples of 50-200 halo power spectra) we are able to provide significantly improved matrices, which are almost indistinguishable from the ones built from much larger samples (thousands of spectra). In order to perform this task we trained convolutional neural networks to denoise the matrices, using in the training process a data set made up entirely of spectra extracted from simple, inexpensive halo simulations (mocks). We then show that the method not only removes the noise in the matrices of the cheap simulation, but it is also able to successfully denoise the matrices of halo power spectra from N-body simulations. We compare the denoised to the other matrices using several metrics, and in all of them they score better, without any signs of spurious artifacts. With the help of the Wishart distribution we derive an analytical extrapolation for the effective sample augmentation allowed by the denoiser. Finally, we show that, by using the denoised matrices, the cosmological parameters can be recovered with nearly the same accuracy as when using matrices built with a sample of 30,000 spectra in the case of the cheap simulations, and with 15,000 spectra in the case of the N-body simulations. Of particular interest is the bias in the Hubble parameter $H_0$, which was significantly reduced after applying the denoiser.

arXiv commentMatches published version; very minor changes wrt V1

References in corpus (49)

• emcee: The MCMC Hammer
• The cosmological simulation code GADGET-2
• LSST: from Science Drivers to Reference Design and Anticipated Data Products
• Toward a halo mass function for precision cosmology: the limits of universality
• The Large Scale Bias of Dark Matter Halos: Numerical Calibration and Model Tests
• Cosmology and fundamental physics with the Euclid satellite
• Why your model parameter confidences might be too optimistic -- unbiased estimation of the inverse covariance matrix
• Extragalactic Science, Cosmology and Galactic Archaeology with the Subaru Prime Focus Spectrograph (PFS)
• The linear power spectrum of observed source number counts
• COLOSSUS: A python toolkit for cosmology, large-scale structure, and dark matter halos
• The Quijote simulations
• An Improved Distance to NGC 4258 and its Implications for the Hubble Constant
• Massive Lossless Data Compression and Multiple Parameter Estimation from Galaxy Spectra
• Parameter inference with estimated covariance matrices
• Learning to Predict the Cosmological Structure Formation
• The Effect of Covariance Estimator Error on Cosmological Parameter Constraints
• Putting the Precision in Precision Cosmology: How accurate should your data covariance matrix be?
• EZmocks: extending the Zel'dovich approximation to generate mock galaxy catalogues with accurate clustering statistics
• PTHalos: A fast method for generating mock galaxy distributions
• Mass Function Predictions Beyond LCDM
• Modelling Baryon Acoustic Oscillations with Perturbation Theory and Stochastic Halo Biasing
• Eigenmode Analysis of Galaxy Redshift Surveys I. Theory and Methods
• The Growth of Correlations in the Matter Power Spectrum
• An accurate tool for the fast generation of dark matter halo catalogs
• The bright-star masks for the HSC-SSP survey
• AI-assisted super-resolution cosmological simulations
• Comparing approximate methods for mock catalogues and covariance matrices II: Power spectrum multipoles
• Comparing approximate methods for mock catalogues and covariance matrices I: correlation function
• Lagrangian perturbation theory at one loop order: successes, failures, and improvements
• Comparing approximate methods for mock catalogues and covariance matrices III: Bispectrum
• Super-resolution emulator of cosmological simulations using deep physical models
• HALOGEN: A tool for fast generation of mock halo catalogues
• Modeling Galaxies in Redshift Space at the Field Level
• Massive data compression for parameter-dependent covariance matrices
• Generating Log-normal Mock Catalog of Galaxies in Redshift Space
• CARPool: fast, accurate computation of large-scale structure statistics by pairing costly and cheap cosmological simulations
• Fast generation of ensembles of cosmological N-body simulations via mode-resampling
• Cosmology with one galaxy?
• BAM: Bias Assignment Method to generate mock catalogs
• AI-assisted super-resolution cosmological simulations II: Halo substructures, velocities and higher order statistics
• ICE-COLA: fast simulations for weak lensing observables
• Non-Linear Matter Power Spectrum Covariance Matrix Errors and Cosmological Parameter Uncertainties
• NECOLA: Towards a Universal Field-level Cosmological Emulator
• CARPool Covariance: Fast, unbiased covariance estimation for large-scale structure observables
• A Bayesian method for combining theoretical and simulated covariance matrices for large-scale structure surveys
• One simulation to have them all: performance of the Bias Assignment Method against N-body simulations
• Fast and Accurate Non-Linear Predictions of Universes with Deep Learning
• BE-HaPPY: Bias Emulator for Halo Power Spectrum including massive neutrinos
• Data Compression and Covariance Matrix Inspection: Cosmic Shear