Fewer Mocks and Less Noise: Reducing the Dimensionality of Cosmological Observables with Subspace Projections

Abstract

Creating accurate and low-noise covariance matrices represents a formidable challenge in modern-day cosmology. We present a formalism to compress arbitrary observables into a small number of bins by projection into a model-specific subspace that minimizes the prior-averaged log-likelihood error. The lower dimensionality leads to a dramatic reduction in covariance matrix noise, significantly reducing the number of mocks that need to be computed. Given a theory model, a set of priors, and a simple model of the covariance, our method works by using singular value decompositions to construct a basis for the observable that is close to Euclidean; by restricting to the first few basis vectors, we can capture almost all the constraining power in a lower-dimensional subspace. Unlike conventional approaches, the method can be tailored for specific analyses and captures non-linearities that are not present in the Fisher matrix, ensuring that the full likelihood can be reproduced. The procedure is validated with full-shape analyses of power spectra from BOSS DR12 mock catalogs, showing that the 96-bin power spectra can be replaced by 12 subspace coefficients without biasing the output cosmology; this allows for accurate parameter inference using only $\sim 100$ mocks. Such decompositions facilitate accurate testing of power spectrum covariances; for the largest BOSS data chunk, we find that: (a) analytic covariances provide accurate models (with or without trispectrum terms); and (b) using the sample covariance from the MultiDark-Patchy mocks incurs a $\sim 0.5\sigma$ shift in ${\Omega }_m$, unless the subspace projection is applied. The method is easily extended to higher order statistics; the $\sim 2000$-bin bispectrum can be compressed into only $\sim 10$ coefficients, allowing for accurate analyses using few mocks and without having to increase the bin sizes.

arXiv comment22 pages, 6 figures. Accepted by Phys. Rev. D

References in corpus (69)

• LSST: from Science Drivers to Reference Design and Anticipated Data Products
• The clustering of galaxies in the completed SDSS-III Baryon Oscillation Spectroscopic Survey: cosmological analysis of the DR12 galaxy sample
• SDSS-III: Massive Spectroscopic Surveys of the Distant Universe, the Milky Way Galaxy, and Extra-Solar Planetary Systems
• The Baryon Oscillation Spectroscopic Survey of SDSS-III
• Wide-Field InfrarRed Survey Telescope-Astrophysics Focused Telescope Assets WFIRST-AFTA 2015 Report
• Karhunen-Loeve eigenvalue problems in cosmology: how should we tackle large data sets?
• Why your model parameter confidences might be too optimistic -- unbiased estimation of the inverse covariance matrix
• Conservative Constraints on Early Cosmology: an illustration of the Monte Python cosmological parameter inference code
• Cosmological Parameters from the BOSS Galaxy Power Spectrum
• The clustering of galaxies in the completed SDSS-III Baryon Oscillation Spectroscopic Survey: Anisotropic galaxy clustering in Fourier-space
• The clustering of galaxies in the SDSS-III Baryon Oscillation Spectroscopic Survey: RSD measurement from the power spectrum and bispectrum of the DR12 BOSS galaxies
• The clustering of galaxies in the SDSS-III Baryon Oscillation Spectroscopic Survey: mock galaxy catalogues for the BOSS Final Data Release
• Massive Lossless Data Compression and Multiple Parameter Estimation from Galaxy Spectra
• The Clustering of Galaxies in the SDSS-III Baryon Oscillation Spectroscopic Survey: Including covariance matrix errors
• Parameter inference with estimated covariance matrices
• The shape of primordial non-Gaussianity and the CMB bispectrum
• Non-linear perturbation theory extension of the Boltzmann code CLASS
• Combining Full-Shape and BAO Analyses of Galaxy Power Spectra: A 1.6% CMB-independent constraint on H0
• The clustering of galaxies in the SDSS-III Baryon Oscillation Spectroscopic Survey: Modeling the clustering and halo occupation distribution of BOSS-CMASS galaxies in the Final Data Release
• The Effect of Covariance Estimator Error on Cosmological Parameter Constraints
• Putting the Precision in Precision Cosmology: How accurate should your data covariance matrix be?
• Effective Sample Size for Importance Sampling based on discrepancy measures
• The Bispectrum: From Theory to Observations
• Modelling Baryon Acoustic Oscillations with Perturbation Theory and Stochastic Halo Biasing
• Blinded challenge for precision cosmology with large-scale structure: results from effective field theory for the redshift-space galaxy power spectrum
• Cosmological Parameters and Neutrino Masses from the Final Planck and Full-Shape BOSS Data
• General CMB and Primordial Bispectrum Estimation I: Mode Expansion, Map-Making and Measures of f_NL
• Gaussian covariance matrices for anisotropic galaxy clustering measurements
• Measuring neutrino masses with large-scale structure: Euclid forecast with controlled theoretical error
• COSEBIs: Extracting the full E-/B-mode information from cosmic shear correlation functions
• Information content of weak lensing power spectrum and bispectrum: including the non-Gaussian error covariance matrix
• Galaxy Power Spectrum Multipoles Covariance in Perturbation Theory
• Automatic physical inference with information maximising neural networks
• Cosmological constraints from BOSS with analytic covariance matrices
• Generalized massive optimal data compression
• Analytic marginalization over CMB calibration and beam uncertainty
• Estimating Cosmological Parameter Covariance
• Comparing approximate methods for mock catalogues and covariance matrices II: Power spectrum multipoles
• The 3-point function in large scale structure: redshift distortions and galaxy bias
• Determining the Hubble Constant without the Sound Horizon: Measurements from Galaxy Surveys
• Comparing approximate methods for mock catalogues and covariance matrices I: correlation function
• An analytic model for the bispectrum of galaxies in redshift space
• Dark matter statistics for large galaxy catalogs: power spectra and covariance matrices
• Comparing approximate methods for mock catalogues and covariance matrices III: Bispectrum
• Recovering Physical Parameters from Galaxy Spectra Using MOPED
• Correlations in the Spatial Power Spectrum Inferred from Angular Clustering: Methods and Application to APM
• An Updated Tomographic Analysis of the Integrated Sachs-Wolfe Effect and implications for Dark Energy
• Perturbation theory approach to predict the covariance matrices of the galaxy power spectrum and bispectrum in redshift space
• Massive data compression for parameter-dependent covariance matrices
• Rapid Separable Analysis of Higher Order Correlators in Large Scale Structure
• Maximal compression of the redshift space galaxy power spectrum and bispectrum
• Combining Probes of Large-Scale Structure with CosmoLike
• Analytic Methods for Cosmological Likelihoods
• Improving the precision matrix for precision cosmology
• Large Covariance Matrices: Smooth Models from the 2-Point Correlation Function
• Data compression in cosmology: A compressed likelihood for Planck data
• Template Bank for Compact Binary Coalescence Searches in Gravitational Wave Data: A General Geometric Placement Algorithm
• Fast Estimation of Gravitational and Primordial Bispectra in Large Scale Structures
• Non-linear shrinkage estimation of large-scale structure covariance
• RascalC: A Jackknife Approach to Estimating Single and Multi-Tracer Galaxy Covariance Matrices
• Precision matrix expansion - efficient use of numerical simulations in estimating errors on cosmological parameters
• Galaxy 2-Point Covariance Matrix Estimation for Next Generation Surveys
• Estimating Covariance Matrices for Two- and Three-Point Correlation Function Moments in Arbitrary Survey Geometries
• Estimating the power spectrum covariance matrix with fewer mock samples
• Large Covariance Matrices: Accurate Models Without Mocks
• Estimating sparse precision matrices
• A Bayesian method for combining theoretical and simulated covariance matrices for large-scale structure surveys
• Extreme data compression while searching for new physics
• Pico: Parameters for the Impatient Cosmologist

Cited by in corpus (38)

• Constraints on Multi-Field Inflation from the BOSS Galaxy Survey
• Probing Parity-Violation with the Four-Point Correlation Function of BOSS Galaxies
• Precision analysis of the redshift-space galaxy bispectrum
• Determining the Hubble Constant without the Sound Horizon: Measurements from Galaxy Surveys
• Cosmology with the Galaxy Bispectrum Multipoles: Optimal Estimation and Application to BOSS Data
• Cosmological constraints from the power spectrum of eBOSS emission line galaxies
• Cosmological constraints without fingers of God
• Cosmology Without Windows: Quadratic Estimators for the Galaxy Power Spectrum
• Cosmology Without Window Functions: Cubic Estimators for the Galaxy Bispectrum
• Going Beyond the Galaxy Power Spectrum: an Analysis of BOSS Data with Wavelet Scattering Transforms
• ENCORE: An $\mathcal{O}(N_{\mathrm{g}}^2)$ Estimator for Galaxy $N$-Point Correlation Functions
• Limits on primordial non-Gaussianities from BOSS galaxy-clustering data
• Determining the Hubble Constant without the Sound Horizon: Perspectives with Future Galaxy Surveys
• Cosmological constraints from the power spectrum of eBOSS quasars
• Cosmology Intertwined: A Review of the Particle Physics, Astrophysics, and Cosmology Associated with the Cosmological Tensions and Anomalies
• Observational constraints on early dark energy
• A Parameter-Masked Mock Data Challenge for Beyond-Two-Point Galaxy Clustering Statistics
• Cosmological Information in Skew Spectra of Biased Tracers in Redshift Space
• Analytic Gaussian Covariance Matrices for Galaxy $N$-Point Correlation Functions
• Semi-analytical covariance matrices for two-point correlation function for DESI 2024 data
• Wavelet Moments for Cosmological Parameter Estimation
• A First Detection of the Connected 4-Point Correlation Function of Galaxies Using the BOSS CMASS Sample
• Light Fields during Inflation from BOSS and Future Galaxy Surveys
• Modal compression of the redshift-space galaxy bispectrum
• Towards cosmological constraints from the compressed modal bispectrum: a robust comparison of real-space bispectrum estimators
• Modeling the Marked Spectrum of Matter and Biased Tracers in Real- and Redshift-Space
• The ACCEL2 Project: Precision Measurements of EFT Parameters and BAO Peak Shifts for the Lyman-$\alpha$ Forest
• Bayesian Control Variates for optimal covariance estimation with pairs of simulations and surrogates
• The halo 3-point correlation function: a methodological analysis
• New approach to template banks of gravitational waves with higher harmonics: Reducing matched-filtering cost by over an order of magnitude
• Bispectrum constraints on Primordial non-Gaussianities with the eBOSS DR16 quasars
• Resonant neutrino self-interactions: Insights from the full shape galaxy power spectrum
• Cosmological inference including massive neutrinos from the matter power spectrum: biases induced by uncertainties in the covariance matrix
• A comparison of shrinkage estimators of the cosmological precision matrix
• Intrinsic alignments of galaxies in multiple projections
• Quijote PNG: The information content of the halo power spectrum and bispectrum
• Cosmology with the Redshift-Space Galaxy Bispectrum Monopole at One-Loop Order
• Three-point intrinsic alignments of galaxies and haloes in the FLAMINGO simulations