A Bayesian method for combining theoretical and simulated covariance matrices for large-scale structure surveys

Abstract

Accurate and precise covariance matrices will be important in enabling planned cosmological surveys to detect new physics. Standard methods imply either the need for many N-body simulations in order to obtain an accurate estimate, or a precise theoretical model. We combine these approaches by constructing a likelihood function conditioned on simulated and theoretical covariances, consistently propagating noise from the finite number of simulations and uncertainty in the theoretical model itself using an informative Inverse-Wishart prior. Unlike standard methods, our approach allows the required number of simulations to be less than the number of summary statistics. We recover the linear 'shrinkage' covariance estimator in the context of a Bayesian data model, and test our marginal likelihood on simulated mock power spectrum estimates. We conduct a thorough investigation into the impact of prior confidence in different choices of covariance models on the quality of model fits and parameter variances. In a simplified setting we find that the number of simulations required can be reduced if one is willing to accept a mild degradation in the quality of model fits, finding that even weakly informative priors can help to reduce the simulation requirements. We identify the correlation matrix of the summary statistics as a key quantity requiring careful modelling. Our approach can be easily generalized to any covariance model or set of summary statistics, and elucidates the role of hybrid estimators in cosmological inference.

21 pages, 11 figures. Minor changes to match the version published in MNRAS

References in corpus (17)

• Why your model parameter confidences might be too optimistic -- unbiased estimation of the inverse covariance matrix
• Simulations of Baryon Acoustic Oscillations II: Covariance matrix of the matter power spectrum
• Survey geometry and the internal consistency of recent cosmic shear measurements
• Bayesian optimisation for likelihood-free cosmological inference
• Cosmological Simulations for Combined-Probe Analyses: Covariance and Neighbour-Exclusion Bias
• Simulations of Weak Gravitational Lensing - II : Including Finite Support Effects in Cosmic Shear Covariance Matrices
• Accurate cosmic shear errors: do we need ensembles of simulations?
• Shrinkage Estimation of the Power Spectrum Covariance Matrix
• Massive data compression for parameter-dependent covariance matrices
• Perturbative approach to covariance matrix of the matter power spectrum
• On the insufficiency of arbitrarily precise covariance matrices: non-Gaussian weak lensing likelihoods
• Matter Power Spectrum Covariance Matrix from the DEUS-PUR ΛCDM simulations: Mass Resolution and non-Gaussian Errors
• A volumetric deep Convolutional Neural Network for simulation of mock dark matter halo catalogues
• Quantifying lost information due to covariance matrix estimation in parameter inference
• Non-linear shrinkage estimation of large-scale structure covariance
• Properties and use of CMB power spectrum likelihoods
• Likelihood Non-Gaussianity in Large-Scale Structure Analyses

Cited by in corpus (19)

• Galaxy Power Spectrum Multipoles Covariance in Perturbation Theory
• Dark Energy Survey Year 3 Results: Covariance Modelling and its Impact on Parameter Estimation and Quality of Fit
• 2D-FFTLog: Efficient computation of real space covariance matrices for galaxy clustering and weak lensing
• Fewer Mocks and Less Noise: Reducing the Dimensionality of Cosmological Observables with Subspace Projections
• Fast and Credible Likelihood-Free Cosmology with Truncated Marginal Neural Ratio Estimation
• Disconnected Covariance of 2-point Functions in Large-Scale Structure
• Cosmological Information in Skew Spectra of Biased Tracers in Redshift Space
• Primordial power spectrum and cosmology from black-box galaxy surveys
• Modal compression of the redshift-space galaxy bispectrum
• Towards cosmological constraints from the compressed modal bispectrum: a robust comparison of real-space bispectrum estimators
• Fitting covariance matrix models to simulations
• Euclid: Covariance of weak lensing pseudo-$C_{\ell }$ estimates. Calculation, comparison to simulations, and dependence on survey geometry
• Bayesian Control Variates for optimal covariance estimation with pairs of simulations and surrogates
• Simulation-based inference has its own Dodelson-Schneider effect (but it knows that it does)
• A comparison of shrinkage estimators of the cosmological precision matrix
• Non-Gaussian likelihood of weak lensing power spectra
• Improving cosmological covariance matrices with machine learning
• Origin & evolution of the Universe
• The Parameter-Level Performance of Covariance Matrix Conditioning in Cosmic Microwave Background Data Analyses