When tension is just a fluctuation: How noisy data affect model comparison
arXiv:2102.09547 · doi:10.1051/0004-6361/202039560
Abstract
Summary statistics of the likelihood, such as the Bayesian evidence, offer a principled way of comparing models and assessing tension between, or within, the results of physical experiments. Noisy realisations of the data induce scatter in these model comparison statistics. For a realistic case of cosmological inference from large-scale structure we show that the logarithm of the Bayes factor attains scatter of order unity, increasing significantly with stronger tension between the models under comparison. We develop an approximate procedure that quantifies the sampling distribution of the evidence at small additional computational cost and apply it to real data to demonstrate the impact of the scatter, which acts to reduce the significance of any model discrepancies. Data compression is highlighted as a potential avenue to suppressing noise in the evidence to negligible levels, with a proof of concept on Planck cosmic microwave background data.
arXiv comment6 pages, 4 figures + appendices; minor changes to match version published in Astronomy and Astrophysics Letters
References in corpus (42)
- MultiNest: an efficient and robust Bayesian inference tool for cosmology and particle physics
- A 2.4% Determination of the Local Value of the Hubble Constant
- Large Magellanic Cloud Cepheid Standards Provide a 1% Foundation for the Determination of the Hubble Constant and Stronger Evidence for Physics Beyond LambdaCDM
- Multimodal nested sampling: an efficient and robust alternative to MCMC methods for astronomical data analysis
- Bayes in the sky: Bayesian inference and model selection in cosmology
- Dark Energy Survey Year 1 Results: Cosmological Constraints from Galaxy Clustering and Weak Lensing
- Tensions between the Early and the Late Universe
- H0LiCOW XIII. A 2.4% measurement of from lensed quasars: tension between early and late-Universe probes
- Planck 2018 results. V. CMB power spectra and likelihoods
- KiDS-450: Cosmological parameter constraints from tomographic weak gravitational lensing
- Importance Nested Sampling and the MultiNest Algorithm
- Karhunen-Loeve eigenvalue problems in cosmology: how should we tackle large data sets?
- Conservative Constraints on Early Cosmology: an illustration of the Monte Python cosmological parameter inference code
- KiDS-1000 Cosmology: Multi-probe weak gravitational lensing and spectroscopic galaxy clustering constraints
- Milky Way Cepheid Standards for Measuring Cosmic Distances and Application to Gaia DR2: Implications for the Hubble Constant
- PolyChord: nested sampling for cosmology
- PolyChord: next-generation nested sampling
- Gravitational Lensing Analysis of the Kilo Degree Survey
- Massive Lossless Data Compression and Multiple Parameter Estimation from Galaxy Spectra
- CFHTLenS revisited: assessing concordance with Planck including astrophysical systematics
- Concordance and Discordance in Cosmology
- KiDS+VIKING-450 and DES-Y1 combined: Cosmology with cosmic shear
- Efficient Cosmological Parameter Estimation from Microwave Background Anisotropies
- Measuring the effective complexity of cosmological models
- On model selection forecasting, Dark Energy and modified gravity
- KiDS+VIKING-450 and DES-Y1 combined: Mitigating baryon feedback uncertainty with COSEBIs
- Quantifying tensions in cosmological parameters: Interpreting the DES evidence ratio
- Bayesian evidence as a tool for comparing datasets
- Constraints on a mixed inflaton and curvaton scenario for the generation of the curvature perturbation
- Quantifying dimensionality: Bayesian cosmological model complexities
- Cosmological discordances: a new measure, marginalization effects, and application to geometry vs growth current data sets
- Forecasting the Bayes factor of a future observation
- H_0 and Odds on Cosmology
- The power of Bayesian evidence in astronomy
- Information Gains from Cosmic Microwave Background Experiments
- Planck confronts large scale structure: methods to quantify discordance
- Consistency tests in cosmology using relative entropy
- Data compression in cosmology: A compressed likelihood for Planck data
- Quantifying Suspiciousness Within Correlated Data Sets
- A new measure of tension between experiments
- Gaussianisation for fast and accurate inference from cosmological data
- A Bayesian quantification of consistency in correlated datasets
Cited by in corpus (10)
- KiDS & Euclid: Cosmological implications of a pseudo angular power spectrum analysis of KiDS-1000 cosmic shear tomography
- Evaluating the prevalence of spurious correlations in pulsar timing array datasets
- Cosmology Intertwined: A Review of the Particle Physics, Astrophysics, and Cosmology Associated with the Cosmological Tensions and Anomalies
- Searching for quasi-periodic oscillations in astrophysical transients using Gaussian processes
- On dataset tensions and signatures of new cosmological physics
- An empirical approach to model selection: weak lensing and intrinsic alignments
- Detecting the periodicity of highly irregularly sampled light-curves with Gaussian processes: the case of SDSSJ025214.67-002813.7
- On The Distribution of Bayesian Evidences
- DHOST gravity in Ultra-diffuse galaxies -- Part II: NGC 1052-DF4 and Dragonfly 44
- DHOST gravity in Ultra-diffuse galaxies -- Part I: the case of NGC1052-DF2