On The Distribution of Bayesian Evidences

Abstract

We look at the distribution of the Bayesian evidence for mock realizations of supernova and baryon acoustic oscillation data. The ratios of Bayesian evidences of different models are often used to perform model selection. The significance of these Bayes factors are then interpreted using scales such as the Jeffreys or Kass \& Raftery scale. First, we demonstrate how to use the evidence itself to validate the model, that is to say how well a model fits the data, regardless of how well other models perform. The basic idea is that if, for some real dataset a model's evidence lies outside the distribution of evidences that result when the same fiducial model that generates the datasets is used for the analysis, then the model in question is robustly ruled out. Further, we show how to assess the significance of a hypothetically computed Bayes factor. We show that the range of the distribution of Bayes factors can greatly depend on the models in question and also the number of data points in the dataset. Thus, we have demonstrated that the significance of Bayes factors needs to be calculated for each unique dataset.

9 pages, 7 figures

References in corpus (21)

• Large Magellanic Cloud Cepheid Standards Provide a 1% Foundation for the Determination of the Hubble Constant and Stronger Evidence for Physics Beyond LambdaCDM
• The Clustering of the SDSS DR7 Main Galaxy Sample I: A 4 per cent Distance Measure at z=0.15
• New Hubble Space Telescope Discoveries of Type Ia Supernovae at z > 1: Narrowing Constraints on the Early Behavior of Dark Energy
• Improved Cosmological Constraints from New, Old and Combined Supernova Datasets
• Spectra and Light Curves of Six Type Ia Supernovae at 0.511 < z < 1.12 and the Union2 Compilation
• The DESI Experiment Part I: Science,Targeting, and Survey Design
• Bayes in the sky: Bayesian inference and model selection in cosmology
• Wide-Field InfrarRed Survey Telescope-Astrophysics Focused Telescope Assets WFIRST-AFTA 2015 Report
• Improved Dark Energy Constraints from ~100 New CfA Supernova Type Ia Light Curves
• The Clustering of the SDSS Main Galaxy Sample II: Mock galaxy catalogues and a measurement of the growth of structure from Redshift Space Distortions at $z=0.15$
• The clustering of galaxies in the completed SDSS-III Baryon Oscillation Spectroscopic Survey: Baryon Acoustic Oscillations in Fourier-space
• Baryon acoustic oscillations from the complete SDSS-III Ly$\alpha$-quasar cross-correlation function at $z=2.4$
• The DESI Experiment Part II: Instrument Design
• KiDS-1000 Cosmology: constraints beyond flat $\Lambda$CDM
• Model Independent Reconstruction of the Expansion History of the Universe and the Properties of Dark Energy
• A comparative study of host galaxy properties between Fast Radio Bursts and stellar transients
• Testing the Mutual Consistency of the Pantheon and SDSS/eBOSS BAO Data Sets with Gaussian Processes
• Dark Energy Survey internal consistency tests of the joint cosmological probes analysis with posterior predictive distributions
• Hunting for wandering massive black holes
• When tension is just a fluctuation: How noisy data affect model comparison
• Model selection and parameter estimation using the iterative smoothing method

Cited by in corpus (4)

• Model selection applied to reconstructions of the Dark Energy
• Testing signatures of phantom crossing through full-shape galaxy clustering analysis
• Evidence for thermal X-ray emission from the synchrotron dominated shocks in Tycho's supernova remnant
• Gaussian discriminators between $\Lambda$CDM and wCDM cosmologies using expansion data