Shrinkage Estimation of the Power Spectrum Covariance Matrix
arXiv:0711.2509 · doi:10.1111/j.1365-2966.2008.13561.x
Abstract
We seek to improve estimates of the power spectrum covariance matrix from a limited number of simulations by employing a novel statistical technique known as shrinkage estimation. The shrinkage technique optimally combines an empirical estimate of the covariance with a model (the target) to minimize the total mean squared error compared to the true underlying covariance. We test this technique on N-body simulations and evaluate its performance by estimating cosmological parameters. Using a simple diagonal target, we show that the shrinkage estimator significantly outperforms both the empirical covariance and the target individually when using a small number of simulations. We find that reducing noise in the covariance estimate is essential for properly estimating the values of cosmological parameters as well as their confidence intervals. We extend our method to the jackknife covariance estimator and again find significant improvement, though simulations give better results. Even for thousands of simulations we still find evidence that our method improves estimation of the covariance matrix. Because our method is simple, requires negligible additional numerical effort, and produces superior results, we always advocate shrinkage estimation for the covariance of the power spectrum and other large-scale structure measurements when purely theoretical modeling of the covariance is insufficient.
arXiv comment9 pages, 7 figures (1 new), MNRAS, accepted. Changes to match accepted version, including an additional explanatory section with 1 figure
References in corpus (8)
- Efficient Computation of CMB anisotropies in closed FRW models
- Stable clustering, the halo model and nonlinear cosmological power spectra
- Galaxy Clusters in Hubble Volume Simulations: Cosmological Constraints from Sky Survey Populations
- Radical Compression of Cosmic Microwave Background Data
- On measuring the covariance matrix of the nonlinear power spectrum from simulations
- Information content in the halo-model dark-matter power spectrum II: Multiple cosmological parameters
- Fast Edge Corrected Measurement of the Two-Point Correlation Function and the Power Spectrum
- Constraining Primordial Non-Gaussianities from the WMAP2 2-1 Cumulant Correlator Power Spectrum
Cited by in corpus (59)
- The VIMOS Public Extragalactic Redshift Survey (VIPERS). Galaxy clustering and redshift-space distortions at z=0.8 in the first data release
- The clustering of galaxies in the SDSS-III Baryon Oscillation Spectroscopic Survey: a large sample of mock galaxy catalogues
- The clustering of galaxies in the completed SDSS-III Baryon Oscillation Spectroscopic Survey: Observational systematics and baryon acoustic oscillations in the correlation function
- CosmoLike - Cosmological Likelihood Analyses for Photometric Galaxy Surveys
- Putting the Precision in Precision Cosmology: How accurate should your data covariance matrix be?
- The Effect of Covariance Estimator Error on Cosmological Parameter Constraints
- Large-Scale Anisotropic Correlation Function of SDSS Luminous Red Galaxies
- Gaussian covariance matrices for anisotropic galaxy clustering measurements
- An accurate tool for the fast generation of dark matter halo catalogs
- Galaxy Power Spectrum Multipoles Covariance in Perturbation Theory
- The VIMOS Public Extragalactic Redshift Survey (VIPERS). Gravity test from the combination of redshift-space distortions and galaxy-galaxy lensing at
- Dark Energy Survey Year 3 Results: Covariance Modelling and its Impact on Parameter Estimation and Quality of Fit
- Estimating Cosmological Parameter Covariance
- The clustering of galaxies in the completed SDSS-III Baryon Oscillation Spectroscopic Survey: theoretical systematics and Baryon Acoustic Oscillations in the galaxy correlation function
- nIFTy Cosmology: Galaxy/halo mock catalogue comparison project on clustering statistics
- Comparing approximate methods for mock catalogues and covariance matrices I: correlation function
- Dark matter statistics for large galaxy catalogs: power spectra and covariance matrices
- A Detection of the Baryon Acoustic Oscillation Features in the SDSS BOSS DR12 Galaxy Bispectrum
- Testing Lorentz symmetry with Lunar Laser Ranging
- Improving fast generation of halo catalogs with higher-order Lagrangian perturbation theory
- Characterizing Signal Loss in the 21 cm Reionization Power Spectrum: A Revised Study of PAPER-64
- The Thousand Asteroid Light Curve Survey
- Perturbation theory approach to predict the covariance matrices of the galaxy power spectrum and bispectrum in redshift space
- Massive data compression for parameter-dependent covariance matrices
- Fast generation of ensembles of cosmological N-body simulations via mode-resampling
- Dark Energy Survey Year 1 Results: galaxy mock catalogues for BAO
- Non-linear shrinkage estimation of large-scale structure covariance
- Precision matrix expansion - efficient use of numerical simulations in estimating errors on cosmological parameters
- Properties and use of CMB power spectrum likelihoods
- BAO from angular clustering: optimization and mitigation of theoretical systematics
- Estimating the power spectrum covariance matrix with fewer mock samples
- Galaxy 2-Point Covariance Matrix Estimation for Next Generation Surveys
- Fast radio bursts trigger aftershocks resembling earthquakes, but not solar flares
- On estimating cosmology-dependent covariance matrices
- Disconnected Covariance of 2-point Functions in Large-Scale Structure
- Cosmological Density Fluctuations on 100Mpc Scales and their ISW Effect
- A Bayesian method for combining theoretical and simulated covariance matrices for large-scale structure surveys
- Removable Matter-Power-Spectrum Covariance from Bias Fluctuations
- The Atacama Cosmology Telescope: Measurement and Analysis of 1D Beams for DR4
- Cosmological Inference from Galaxy-Clustering Power Spectrum: Gaussianization and Covariance Decomposition
- RedMaPPer: Evolution and Mass Dependence of the Conditional Luminosity Functions of Red Galaxies in Galaxy Clusters
- Accurate halo-galaxy mocks from automatic bias estimation and particle mesh gravity solvers
- Robustness of the covariance matrix for galaxy clustering measurements
- Detecting Baryon Acoustic Oscillations
- Fitting covariance matrix models to simulations
- Indra: a Public Computationally Accessible Suite of Cosmological -body Simulations
- Optimal Weights For Measuring Redshift Space Distortions in Multi-tracer Galaxy Catalogues
- Bayesian Control Variates for optimal covariance estimation with pairs of simulations and surrogates
- Uncertainty in 2-point correlation function estimators and BAO detection in SDSS DR7
- Non-Gaussian error bars in galaxy surveys -- II
- The halo 3-point correlation function: a methodological analysis
- Cosmological inference including massive neutrinos from the matter power spectrum: biases induced by uncertainties in the covariance matrix
- Approximate methods for the generation of dark matter halo catalogs in the age of precision cosmology
- Euclid: Fast two-point correlation function covariance through linear construction
- A comparison of shrinkage estimators of the cosmological precision matrix
- Covariances of Galaxy Stellar Mass Functions and Correlation Functions
- Euclid: Modelling observational effects in cluster counts and cluster clustering
- Super sample covariance and the volume scaling of galaxy survey covariance matrices
- Replicating weak-lensing summary-statistic covariances with normalizing flows