Quantifying lost information due to covariance matrix estimation in parameter inference

Abstract

Parameter inference with an estimated covariance matrix systematically loses information due to the remaining uncertainty of the covariance matrix. Here, we quantify this loss of precision and develop a framework to hypothetically restore it, which allows to judge how far away a given analysis is from the ideal case of a known covariance matrix. We point out that it is insufficient to estimate this loss by debiasing a Fisher matrix as previously done, due to a fundamental inequality that describes how biases arise in non-linear functions. We therefore develop direct estimators for parameter credibility contours and the figure of merit. We apply our results to DES Science Verification weak lensing data, detecting a 10% loss of information that increases their credibility contours. No significant loss of information is found for KiDS. For a Euclid-like survey, with about 10 nuisance parameters we find that 2900 simulations are sufficient to limit the systematically lost information to 1%, with an additional uncertainty of about 2%. Without any nuisance parameters 1900 simulations are sufficient to only lose 1% of information. We also derive an estimator for the Fisher matrix of the unknown true covariance matrix, two estimators of its inverse with different physical meanings, and an estimator for the optimally achievable figure of merit. The formalism here quantifies the gains to be made by running more simulated datasets, allowing decisions to be made about numbers of simulations in an informed way.

Figure 6 holds for all surveys; replacement to match published version

References in corpus (12)

• MultiNest: an efficient and robust Bayesian inference tool for cosmology and particle physics
• Karhunen-Loeve eigenvalue problems in cosmology: how should we tackle large data sets?
• Why your model parameter confidences might be too optimistic -- unbiased estimation of the inverse covariance matrix
• CFHTLenS tomographic weak lensing cosmological parameter constraints: Mitigating the impact of intrinsic galaxy alignments
• The Clustering of Galaxies in the SDSS-III Baryon Oscillation Spectroscopic Survey: Including covariance matrix errors
• Parameter inference with estimated covariance matrices
• The Effect of Covariance Estimator Error on Cosmological Parameter Constraints
• Putting the Precision in Precision Cosmology: How accurate should your data covariance matrix be?
• Estimating Cosmological Parameter Covariance
• Simulations of Weak Gravitational Lensing - II : Including Finite Support Effects in Cosmic Shear Covariance Matrices
• Matter Power Spectrum Covariance Matrix from the DEUS-PUR ΛCDM simulations: Mass Resolution and non-Gaussian Errors
• Simulations of Wide-Field Weak Lensing Surveys II: Covariance Matrix of Real Space Correlation Functions

Cited by in corpus (33)

• Weak lensing for precision cosmology
• Dark Energy Survey Year 3 Results: Covariance Modelling and its Impact on Parameter Estimation and Quality of Fit
• 2D-FFTLog: Efficient computation of real space covariance matrices for galaxy clustering and weak lensing
• Cosmological constraints from noisy convergence maps through deep learning
• Massive data compression for parameter-dependent covariance matrices
• Cosmology Without Windows: Quadratic Estimators for the Galaxy Power Spectrum
• Generating Log-normal Mock Catalog of Galaxies in Redshift Space
• Toward a robust inference method for the galaxy bispectrum: likelihood function and model selection
• Joint analysis of anisotropic power spectrum, bispectrum and trispectrum: application to N-body simulations
• Euclid Preparation. XXVIII. Forecasts for ten different higher-order weak lensing statistics
• On the insufficiency of arbitrarily precise covariance matrices: non-Gaussian weak lensing likelihoods
• Mock galaxy shape catalogs in the Subaru Hyper Suprime-Cam Survey
• The skewed weak lensing likelihood: why biases arise, despite data and theory being sound
• Precision matrix expansion - efficient use of numerical simulations in estimating errors on cosmological parameters
• Probing dark energy with tomographic weak-lensing aperture mass statistics
• ICE-COLA: fast simulations for weak lensing observables
• On the accuracy and precision of correlation functions and field-level inference in cosmology
• Covariance of the galaxy angular power spectrum with the halo model
• Fast Generation of Covariance Matrices for Weak Lensing
• Debiasing inference with approximate covariance matrices and other unidentified biases
• A Bayesian method for combining theoretical and simulated covariance matrices for large-scale structure surveys
• It takes two to know one: Computing accurate one-point PDF covariances from effective two-point PDF models
• Integrated trispectrum detection from BOSS DR12 NGC CMASS
• Extreme data compression while searching for new physics
• A blinding solution for inference from astronomical data
• Euclid: Covariance of weak lensing pseudo-$C_{\ell }$ estimates. Calculation, comparison to simulations, and dependence on survey geometry
• Testing the accuracy of likelihoods for cluster abundance cosmology
• Cosmological Model Parameter Dependence of the Matter Power Spectrum Covariance from the DEUS-PUR $Cosmo$ Simulations
• Cosmological inference including massive neutrinos from the matter power spectrum: biases induced by uncertainties in the covariance matrix
• Combined $13\times 2$-point analysis of the Cosmic Microwave Background and Large-Scale Structure: implications for the $S_8$-tension and neutrino mass constraints
• Matching Bayesian and frequentist coverage probabilities when using an approximate data covariance matrix
• Origin & evolution of the Universe
• Dark Energy Survey Year 3 Results: Three-Point Shear Correlations and Mass Aperture Moments