A comparison of shrinkage estimators of the cosmological precision matrix

Abstract

The determination of the covariance matrix and its inverse, the precision matrix, is critical in the statistical analysis of cosmological measurements. The covariance matrix is typically estimated with a limited number of simulations at great computational cost before inversion into the precision matrix; therefore, it can be ill-conditioned and overly noisy when the sample size $n$ used for estimation is not much larger than the data vector dimension. In this work, we consider a class of methods known as shrinkage estimation for the precision matrix, which combines an empirical estimate with a target that is either analytical or stochastic. These methods include linear and non-linear shrinkage applied to the covariance matrix (the latter represented by the so-called NERCOME estimator), and the direct linear shrinkage estimation of the precision matrix which we introduce in a cosmological setting. Using Bayesian parameter inference as well as metrics like matrix loss functions and the eigenvalue spectrum, we compare their performance against the standard sample estimator with varying sample size $n$. We have found the shrinkage estimators to significantly improve the posterior distribution at low $n$, especially for the linear shrinkage estimators either inverted from the covariance matrix or applied directly to the precision matrix, with an empirical target constructed from the sample estimate. Our results should be particularly relevant to the analyses of Stage-IV spectroscopic galaxy surveys such as the Dark Energy Spectroscopic Instrument (DESI) and Euclid, whose statistical power can be limited by the computational cost of obtaining an accurate precision matrix estimate.

arXiv comment14 pages, 11 figures, 4 tables. Accepted for publication in MNRAS

References in corpus (44)

• SciPy 1.0--Fundamental Algorithms for Scientific Computing in Python
• Array Programming with NumPy
• emcee: The MCMC Hammer
• Planck 2015 results. XIII. Cosmological parameters
• The Eleventh and Twelfth Data Releases of the Sloan Digital Sky Survey: Final Data from SDSS-III
• Baryonic Features in the Matter Transfer Function
• Why your model parameter confidences might be too optimistic -- unbiased estimation of the inverse covariance matrix
• Nonlinear shrinkage estimation of large-dimensional covariance matrices
• The Cosmological Analysis of the SDSS/BOSS data from the Effective Field Theory of Large-Scale Structure
• The Early Data Release of the Dark Energy Spectroscopic Instrument
• The clustering of galaxies in the SDSS-III Baryon Oscillation Spectroscopic Survey: mock galaxy catalogues for the BOSS Final Data Release
• The VIMOS Public Extragalactic Redshift Survey (VIPERS). Galaxy clustering and redshift-space distortions at z=0.8 in the first data release
• nbodykit: an open-source, massively parallel toolkit for large-scale structure
• The Clustering of Galaxies in the SDSS-III Baryon Oscillation Spectroscopic Survey: Including covariance matrix errors
• Parameter inference with estimated covariance matrices
• Putting the Precision in Precision Cosmology: How accurate should your data covariance matrix be?
• The Effect of Covariance Estimator Error on Cosmological Parameter Constraints
• On measuring the covariance matrix of the nonlinear power spectrum from simulations
• Gaussian covariance matrices for anisotropic galaxy clustering measurements
• Galaxy Power Spectrum Multipoles Covariance in Perturbation Theory
• Cosmological constraints from BOSS with analytic covariance matrices
• Estimating Cosmological Parameter Covariance
• Comparing approximate methods for mock catalogues and covariance matrices II: Power spectrum multipoles
• Comparing approximate methods for mock catalogues and covariance matrices III: Bispectrum
• zeus: A Python implementation of Ensemble Slice Sampling for efficient Bayesian parameter inference
• Shrinkage Estimation of the Power Spectrum Covariance Matrix
• Accelerating astronomical and cosmological inference with Preconditioned Monte Carlo
• Perturbation theory approach to predict the covariance matrices of the galaxy power spectrum and bispectrum in redshift space
• Fewer Mocks and Less Noise: Reducing the Dimensionality of Cosmological Observables with Subspace Projections
• Performance of internal Covariance Estimators for Cosmic Shear Correlation Functions
• Optimal Linear Shrinkage Estimator for Large Dimensional Precision Matrix
• Perturbative approach to covariance matrix of the matter power spectrum
• Euclid Preparation. XXVIII. Forecasts for ten different higher-order weak lensing statistics
• On the insufficiency of arbitrarily precise covariance matrices: non-Gaussian weak lensing likelihoods
• Precision matrix expansion - efficient use of numerical simulations in estimating errors on cosmological parameters
• Non-linear shrinkage estimation of large-scale structure covariance
• RascalC: A Jackknife Approach to Estimating Single and Multi-Tracer Galaxy Covariance Matrices
• Creating Jackknife and Bootstrap estimates of the covariance matrix for the two-point correlation function
• Non-Linear Matter Power Spectrum Covariance Matrix Errors and Cosmological Parameter Uncertainties
• Large Covariance Matrices: Accurate Models Without Mocks
• Disconnected Covariance of 2-point Functions in Large-Scale Structure
• A Bayesian method for combining theoretical and simulated covariance matrices for large-scale structure surveys
• Fitting covariance matrix models to simulations
• Cosmological inference including massive neutrinos from the matter power spectrum: biases induced by uncertainties in the covariance matrix

Cited by in corpus (1)

• The Atacama Cosmology Telescope: Semi-Analytic Covariance Matrices for the DR6 CMB Power Spectra