Myths and Truths Concerning Estimation of Power Spectra
arXiv:astro-ph/0307515 · doi:10.1111/j.1365-2966.2004.07530.x
Abstract
It is widely believed that maximum likelihood estimators must be used to provide optimal estimates of power spectra. Since such estimators require require of order N_d^3 operations they are computationally prohibitive for N_d greater than a few tens of thousands. Because of this, a large and inhomogeneous literature exists on approximate methods of power spectrum estimation. These range from manifestly sub-optimal, but computationally fast methods, to near optimal but computationally expensive methods. Furthermore, much of this literature concentrates on the power spectrum estimates rather than the equally important problem of deriving an accurate covariance matrix. In this paper, I consider the problem of estimating the power spectrum of cosmic microwave background (CMB) anisotropies from large data sets. Various analytic results on power spectrum estimators are derived, or collated from the literature, and tested against numerical simulations. An unbiased hybrid estimator is proposed that combines a maximum likelihood estimator at low multipoles and pseudo-C_\ell estimates at high multipoles. The hybrid estimator is computationally fast, nearly optimal over the full range of multipoles, and returns an accurate and nearly diagonal covariance matrix for realistic experimental configurations (provided certain conditions on the noise properties of the experiment are satisfied). It is argued that, in practice, computationally expensive methods that approximate the N_d^3 maximum likelihood solution are unlikely to improve on the hybrid estimator, and may actually perform worse. The results presented here can be generalised to CMB polarization and to power spectrum estimation using other types of data, such as galaxy clustering and weak gravitational lensing.
27 pages, 15 figures, MNRAS in press. Resubmission matches accepted version
References in corpus (9)
- First Year Wilkinson Microwave Anisotropy Probe (WMAP) Observations: Determination of Cosmological Parameters
- First Year Wilkinson Microwave Anisotropy Probe (WMAP) Observations: Preliminary Maps and Basic Results
- First Year Wilkinson Microwave Anisotropy Probe (WMAP) Observations: Angular Power Spectrum
- First Year Wilkinson Microwave Anisotropy Probe (WMAP) Observations: Parameter Estimation Methodology
- Improved Measurement of the Angular Power Spectrum of Temperature Anisotropy in the CMB from Two New Analyses of BOOMERANG Observations
- The Statistical Significance of the Low CMB Mulitipoles
- Gabor Transforms on the Sphere with Applications to CMB Power Spectrum Estimation
- Fast CMB Power Spectrum Estimation of Temperature and Polarisation with Gabor Transforms
- Fast Power Spectrum Estimation
Cited by in corpus (169)
- Planck 2015 results. XIII. Cosmological parameters
- Planck 2013 results. XVI. Cosmological parameters
- Planck 2018 results. V. CMB power spectra and likelihoods
- Three-Year Wilkinson Microwave Anisotropy Probe (WMAP) Observations: Temperature Analysis
- Planck 2015 results. XI. CMB power spectra, likelihoods, and robustness of parameters
- Planck 2013 results. XV. CMB power spectra and likelihood
- A unified pseudo- framework
- Planck intermediate results. XLVI. Reduction of large-scale systematic effects in HFI polarization maps and estimation of the reionization optical depth
- Correlation of CMB with large-scale structure: I. ISW Tomography and Cosmological Implications
- Joint Bayesian component separation and CMB power spectrum estimation
- Likelihood Analysis of CMB Temperature and Polarization Power Spectra
- Fast estimation of polarization power spectra using correlation functions
- Global, Exact Cosmic Microwave Background Data Analysis Using Gibbs Sampling
- Cosmological baryonic and matter densities from 600,000 SDSS Luminous Red Galaxies with photometric redshifts
- Xspect, estimation of the angular power spectrum by computing cross-power spectra with analytical error bars
- On using angular cross-correlations to determine source redshift distributions
- Planck pre-launch status: the Planck-LFI programme
- Algorithms for bispectra: forecasting, optimal analysis, and simulation
- Cosmological constraints from unWISE and Planck CMB lensing tomography
- Cosmological weak lensing with the HST GEMS survey
- CMB temperature and polarisation pseudo-Cl estimators and covariances
- Asymmetric Beams and CMB Statistical Anisotropy
- Dark Energy Survey Year 3 Results: A 2.7% measurement of Baryon Acoustic Oscillation distance scale at redshift 0.835
- Cosmological parameters from WMAP 5-year temperature maps
- The significance of the integrated Sachs-Wolfe effect revisited
- Dark Energy Survey Year 3 Results: Covariance Modelling and its Impact on Parameter Estimation and Quality of Fit
- Spectral estimation on a sphere in geophysics and cosmology
- The growth of density perturbations in the last 10 billion years from tomographic large-scale structure data
- Estimating the large-scale angular power spectrum in the presence of systematics: a case study of Sloan Digital Sky Survey quasars
- Upper Bound of Neutrino Masses from Combined Cosmological Observations and Particle Physics Experiments
- Exact likelihood evaluations and foreground marginalization in low resolution WMAP data
- Origin of Small-Scale Anisotropies in Galactic Cosmic Rays
- Observation of Small-scale Anisotropy in the Arrival Direction Distribution of TeV Cosmic Rays with HAWC
- unWISE tomography of Planck CMB lensing
- Error analysis of quadratic power spectrum estimates for CMB polarization: sampling covariance
- FASTLens (FAst STatistics for weak Lensing) : Fast method for Weak Lensing Statistics and map making
- An Updated Tomographic Analysis of the Integrated Sachs-Wolfe Effect and implications for Dark Energy
- Accurate cosmic shear errors: do we need ensembles of simulations?
- Unbiased Estimation of an Angular Power Spectrum
- Tomographic galaxy clustering with the Subaru Hyper Suprime-Cam first year public data release
- Fast optimal CMB power spectrum estimation with Hamiltonian sampling
- Complete super-sample lensing covariance in the response approach
- Hybrid Estimation of CMB Polarization Power Spectra
- Calibration errors unleashed: effects on cosmological parameters and requirements for large-scale structure surveys
- Cosmological parameters, shear maps and power spectra from CFHTLenS using Bayesian hierarchical inference
- Combining power spectrum and bispectrum measurements to detect oscillatory features
- Tomography of the Fermi-LAT gamma-ray diffuse extragalactic signal via cross-correlations with galaxy catalogs
- Cosmological Measurements from Angular Power Spectra Analysis of BOSS DR12 Tomography
- Tomographic measurement of the intergalactic gas pressure through galaxy-tSZ cross-correlations
- Measuring the Integrated Sachs-Wolfe Effect
- Dark matter searches in the gamma-ray extragalactic background via cross-correlations with galaxy catalogues
- Cosmology Without Windows: Quadratic Estimators for the Galaxy Power Spectrum
- Cosmological Parameter Constraints as Derived from the Wilkinson Microwave Anisotropy Probe Data via Gibbs Sampling and the Blackwell-Rao Estimator
- Exploiting the full potential of photometric quasar surveys: Optimal power spectra through blind mitigation of systematics
- Cosmic shear power spectra in practice
- Disconnected pseudo- covariances for projected large-scale structure data
- CMB power spectrum estimation using wavelets
- All-Sky Measurement of the Anisotropy of Cosmic Rays at 10 TeV and Mapping of the Local Interstellar Magnetic Field
- Large-Angle Correlations in the Cosmic Microwave Background
- Polyspectra searches for sharp oscillatory features in cosmic microwave sky data
- Probing cosmology with weak lensing selected clusters I: Halo approach and all-sky simulations
- Measurement of CIB power spectra over large sky areas from Planck HFI maps
- Hefty enhancement of cosmological constraints from the DES Y1 data using a Hybrid Effective Field Theory approach to galaxy bias
- New Optimised Estimators for the Primordial Trispectrum
- Evidence of cross-correlation between the CMB lensing and the gamma-ray sky
- Cross-correlating the gamma-ray sky with catalogs of galaxy clusters
- KiDS & Euclid: Cosmological implications of a pseudo angular power spectrum analysis of KiDS-1000 cosmic shear tomography
- The Knotted Sky I: Planck constraints on the primordial power spectrum
- New estimates of the CMB angular power spectra from the WMAP 5 yrs low resolution data
- Integrated Sachs Wolfe Effect and Rees Sciama Effect
- Cosmic shear with small scales: DES-Y3, KiDS-1000 and HSC-DR1
- Likelihood methods for the combined analysis of CMB temperature and polarisation power spectra
- Cosmological constraints from a joint analysis of cosmic microwave background and spectroscopic tracers of the large-scale structure
- Dark Energy Survey: A 2.1% measurement of the angular Baryonic Acoustic Oscillation scale at redshift =0.85 from the final dataset
- Separating E and B types of polarization on an incomplete sky
- The cut-sky cosmic microwave background is not anomalous
- Testing a direction-dependent primordial power spectrum with observations of the Cosmic Microwave Background
- Comparison of map-making algorithms for CMB experiments
- Cross-Correlation of Planck CMB Lensing with DESI-Like LRGs
- The Atacama Cosmology Telescope DR6 and DESI: Structure formation over cosmic time with a measurement of the cross-correlation of CMB Lensing and Luminous Red Galaxies
- Performance forecasts for the primordial gravitational wave detection pipelines for AliCPT-1
- Unresolved Gamma-Ray Sky through its Angular Power Spectrum
- Determination of miscalibrated polarization angles from observed CMB and foreground power spectra: Application to partial-sky observation
- Properties and use of CMB power spectrum likelihoods
- Dark Energy Survey Year 1 Results: Measurement of the Galaxy Angular Power Spectrum
- Likelihood Non-Gaussianity in Large-Scale Structure Analyses
- Fast Edge Corrected Measurement of the Two-Point Correlation Function and the Power Spectrum
- Using inpainting to construct accurate cut-sky CMB estimators
- Cross-correlation of galaxies and galaxy clusters in the Sloan Digital Sky Survey and the importance of non-Poissonian shot noise
- Likelihood methods for CMB experiments
- Quality over Quantity: Optimizing pulsar timing array analysis for stochastic and continuous gravitational wave signals
- Quadratic estimator for CMB cross-correlation
- Probing galaxy bias and intergalactic gas pressure with KiDS Galaxies-tSZ-CMB lensing cross-correlations
- Secondary non-Gaussianity and Cross-Correlation Analysis
- Improving Small-Scale CMB Lensing Reconstruction
- The Integrated Sachs Wolfe effect: unWISE and Planck constraints on Dynamical Dark Energy
- Higher Order Statistics for Three-dimensional Shear and Flexion
- Tomographic imaging of the Fermi-LAT gamma-ray sky through cross-correlations: A wider and deeper look
- CMB power spectrum estimation using noncircular beams
- Bias in low-multipole CMB reconstructions
- Assessing Consistency Between WMAP 9-year and Planck 2015 Temperature Power Spectra
- Characterizing the local gamma-ray Universe via angular cross-correlations
- In-flight polarization angle calibration for LiteBIRD: blind challenge and cosmological implications
- Ephemeris Errors and the Gravitational Wave Signal: Harmonic Mode Coupling in Pulsar Timing Array Searches
- A comparison of CMB Angular Power Spectrum Estimators at Large Scales: the TT case
- Angular Power Spectra with Finite Counts
- Optimal constraints on primordial gravitational waves from the lensed CMB
- An alternative validation strategy for the Planck cluster catalog and -distortion maps
- Probing the statistical properties of CMB -mode polarization through Minkowski Functionals
- A hybrid approach to CMB lensing reconstruction on all-sky intensity maps
- Searching for gamma-ray emission from galaxy clusters at low redshift
- Ergodicity and Gaussianity for Spherical Random Fields
- Preferred axis of CMB parity asymmetry in the masked maps
- Effects of Destriping Errors on Estimates of the CMB Power Spectrum
- A New Maximum-Likelihood Technique for Reconstructing Cosmic-Ray Anisotropy at All Angular Scales
- Cosmic microwave background power spectrum estimation with the destriping technique
- CMB component separation in the pixel domain
- Impact of survey geometry and super-sample covariance on future photometric galaxy surveys
- Analytical marginalisation over photometric redshift uncertainties in cosmic shear analyses
- Constraints on non-resonant photon-axion conversion from the Planck satellite data
- Hybrid-basis inference for large-scale galaxy clustering: combining spherical and Cartesian Fourier analyses
- Real-world CMB lensing quadratic estimator power spectrum response
- Constraining gravity with a new precision estimator using Planck + SDSS BOSS
- Catalog-based pseudo-s
- Constraining the galaxy-halo connection of infrared-selected unWISE galaxies with galaxy clustering and galaxy-CMB lensing power spectra
- Data analysis methods for the cosmic microwave background
- Likelihood, Fisher information, and systematics of cosmic microwave background experiments
- ECLIPSE: a fast Quadratic Maximum Likelihood estimator for CMB intensity and polarization power spectra
- Cosmo++: An Object-Oriented C++ Library for Cosmology
- Effects of Destriping Errors on CMB Polarisation Power Spectra and and Pixel Noise Covariances
- Euclid: Covariance of weak lensing pseudo- estimates. Calculation, comparison to simulations, and dependence on survey geometry
- A Cross-correlation Study between IceCube Neutrino Events and the Fermi Unresolved Gamma-ray Sky
- Exact joint likelihood of pseudo- estimates from correlated Gaussian cosmological fields
- The Cosmic Microwave Background Radiation Power Spectrum as a Random Bit Generator for Symmetric and Asymmetric-Key Cryptography
- Weak Lensing Skew-Spectrum
- Cross-correlation between CMB lensing potential and galaxy catalogues from HELP
- Constraining the Halo Mass of Damped Ly Absorption Systems (DLAs) at using the Quasar-CMB Lensing Cross-correlation
- Self-consistent redshift estimation using correlation functions without a spectroscopic reference sample
- Impact of modelling foreground uncertainties on future CMB polarization satellite experiments
- CMB power spectrum estimation with non-circular beam and incomplete sky coverage
- Cross Correlation of the Extragalactic Gamma-ray Background with Thermal Sunyaev-Zel'dovich Effect in the Cosmic Microwave Background
- Impact of Galactic dust non-Gaussianity on searches for B-modes from inflation
- Degree-scale anomalies in the CMB: localizing the first peak dip to a small patch of the north ecliptic sky
- Non-Circular beam correction to the CMB power spectrum
- The Atacama Cosmology Telescope: Semi-Analytic Covariance Matrices for the DR6 CMB Power Spectra
- Fast scalar quadratic maximum likelihood estimators for the CMB B-mode power spectrum
- Accurate estimation of angular power spectra for maps with correlated masks
- Scalar quadratic maximum likelihood estimators for the CMB cross power spectrum
- Foreground Maps in WMAP frequency bands
- Statistical imprints of CMB B-type polarization leakage in an incomplete sky survey analysis
- Towards efficient and optimal analysis of CMB anisotropies on a masked sky
- Cosmic star formation history with tomographic cosmic infrared background-galaxy cross-correlation
- Cosmological Parameters from a re-analysis of the WMAP-7 low resolution maps
- Searching for All-Scale Anisotropies in the Arrival Directions of Cosmic Rays above the Ankle
- Testing Quadratic Maximum Likelihood estimators for forthcoming Stage-IV weak lensing surveys
- Smoothing methods comparison for CMB E- and B-mode separation
- Higher-Order Spectra of Weak Lensing Convergence Maps in Parameterized Theories of Modified Gravity
- Group Representations and High-Resolution Central Limit Theorems for Subordinated Spherical Random Fields
- Galactic foreground constraints on primordial -mode detection for ground-based experiments
- An Optimally Weighted Estimator of the Linear Power Spectrum Disentangling the Growth of Density Perturbations Across Galaxy Surveys
- The linear bias of radio galaxies at z~0.3 via cosmic microwave background lensing
- Model-independent constraints on clustering and growth of cosmic structures from BOSS DR12 galaxies in harmonic space
- Needlet estimation of cross-correlation between CMB lensing maps and LSS
- Robust Likelihoods for Inflationary Gravitational Waves from Maps of Cosmic Microwave Background Polarization
- Pixelization effects in cosmic shear angular power spectra
- Accurate and efficient likelihood modeling for large-scale CMB data
- Statical properties of CMB B-mode polarisation in a partial sky analysis
- Fast computation of temperature and polarization coupling matrices
- Direct power spectral density estimation from structure functions without Fourier transforms