Inadequacy of internal covariance estimation for super-sample covariance

Abstract

We give an analytical interpretation of how subsample-based internal covariance estimators lead to biased estimates of the covariance, due to underestimating the super-sample covariance (SSC). This includes the jackknife and bootstrap methods as estimators for the full survey area, and subsampling as an estimator of the covariance of subsamples. The limitations of the jackknife covariance have been previously presented in the literature because it is effectively a rescaling of the covariance of the subsample area. However we point out that subsampling is also biased, but for a different reason: the subsamples are not independent, and the corresponding lack of power results in SSC underprediction. We develop the formalism in the case of cluster counts that allows the bias of each covariance estimator to be exactly predicted. We find significant effects for a small-scale area or when a low number of subsamples is used, with auto-redshift biases ranging from 0.4% to 15% for subsampling and from 5% to 75% for jackknife covariance estimates. The cross-redshift covariance is even more affected; biases range from 8% to 25% for subsampling and from 50% to 90% for jackknife. Owing to the redshift evolution of the probe, the covariances cannot be debiased by a simple rescaling factor, and an exact debiasing has the same requirements as the full SSC prediction. These results thus disfavour the use of internal covariance estimators on data itself or a single simulation, leaving analytical prediction and simulations suites as possible SSC predictors.

7 pages, 6 figures. Updated to match version published in A&A

References in corpus (4)

• Why your model parameter confidences might be too optimistic -- unbiased estimation of the inverse covariance matrix
• Probing dark energy with cluster counts and cosmic shear power spectra: including the full covariance
• Comparing halo bias from abundance and clustering
• Linear and non-linear bias: predictions vs. measurements

Cited by in corpus (12)

• Galaxy Power Spectrum Multipoles Covariance in Perturbation Theory
• Cosmological constraints from noisy convergence maps through deep learning
• The skewed weak lensing likelihood: why biases arise, despite data and theory being sound
• Super-sample covariance approximations and partial sky coverage
• Fast and easy super-sample covariance of large scale structure observables
• Covariance of the galaxy angular power spectrum with the halo model
• Debiasing inference with approximate covariance matrices and other unidentified biases
• Impact of survey geometry and super-sample covariance on future photometric galaxy surveys
• The impact of braiding covariance and in-survey covariance on next-generation galaxy surveys
• The 2-point correlation function covariance with fewer mocks
• Efficient computation of the super-sample covariance for stage IV galaxy surveys
• Methodological refinement of the submillimeter galaxy magnification bias. II. Cosmological analysis with tomography