A comparison of shrinkage estimators of the cosmological precision matrix

Abstract

The determination of the covariance matrix and its inverse, the precision matrix, is critical in the statistical analysis of cosmological measurements. The covariance matrix is typically estimated with a limited number of simulations at great computational cost before inversion into the precision matrix; therefore, it can be ill-conditioned and overly noisy when the sample size $n$ used for estimation is not much larger than the data vector dimension. In this work, we consider a class of methods known as shrinkage estimation for the precision matrix, which combines an empirical estimate with a target that is either analytical or stochastic. These methods include linear and non-linear shrinkage applied to the covariance matrix (the latter represented by the so-called NERCOME estimator), and the direct linear shrinkage estimation of the precision matrix which we introduce in a cosmological setting. Using Bayesian parameter inference as well as metrics like matrix loss functions and the eigenvalue spectrum, we compare their performance against the standard sample estimator with varying sample size $n$. We have found the shrinkage estimators to significantly improve the posterior distribution at low $n$, especially for the linear shrinkage estimators either inverted from the covariance matrix or applied directly to the precision matrix, with an empirical target constructed from the sample estimate. Our results should be particularly relevant to the analyses of Stage-IV spectroscopic galaxy surveys such as the Dark Energy Spectroscopic Instrument (DESI) and Euclid, whose statistical power can be limited by the computational cost of obtaining an accurate precision matrix estimate.

14 pages, 11 figures, 4 tables. Accepted for publication in MNRAS

References in corpus (16)

• Array Programming with NumPy
• Why your model parameter confidences might be too optimistic -- unbiased estimation of the inverse covariance matrix
• Nonlinear shrinkage estimation of large-dimensional covariance matrices
• The Early Data Release of the Dark Energy Spectroscopic Instrument
• Cosmological constraints from BOSS with analytic covariance matrices
• zeus: A Python implementation of Ensemble Slice Sampling for efficient Bayesian parameter inference
• Shrinkage Estimation of the Power Spectrum Covariance Matrix
• Fewer Mocks and Less Noise: Reducing the Dimensionality of Cosmological Observables with Subspace Projections
• Accelerating astronomical and cosmological inference with Preconditioned Monte Carlo
• Perturbative approach to covariance matrix of the matter power spectrum
• Euclid Preparation. XXVIII. Forecasts for ten different higher-order weak lensing statistics
• On the insufficiency of arbitrarily precise covariance matrices: non-Gaussian weak lensing likelihoods
• Non-linear shrinkage estimation of large-scale structure covariance
• Creating Jackknife and Bootstrap estimates of the covariance matrix for the two-point correlation function
• Fitting covariance matrix models to simulations
• Cosmological inference including massive neutrinos from the matter power spectrum: biases induced by uncertainties in the covariance matrix

Cited by in corpus (1)

• The Atacama Cosmology Telescope: Semi-Analytic Covariance Matrices for the DR6 CMB Power Spectra