Is the Jeffreys' scale a reliable tool for Bayesian model comparison in cosmology?

Abstract

We are entering an era where progress in cosmology is driven by data, and alternative models will have to be compared and ruled out according to some consistent criterium. The most conservative and widely used approach is Bayesian model comparison. In this paper we explicitly calculate the Bayes factors for all models that are linear with respect to their parameters. We do this in order to test the so called Jeffreys' scale and determine analytically how accurate its predictions are in a simple case where we fully understand and can calculate everything analytically. We also discuss the case of nested models, e.g. one with $M_1$ and another with $M_2\supset M_1$ parameters and we derive analytic expressions for both the Bayes factor and the Figure of Merit, defined as the inverse area of the model parameter's confidence contours. With all this machinery and the use of an explicit example we demonstrate that the threshold nature of Jeffreys' scale is not a "one size fits all" reliable tool for model comparison and that it may lead to biased conclusions. Furthermore, we discuss the importance of choosing the right basis in the context of models that are linear with respect to their parameters and how that basis affects the parameter estimation and the derived constraints.

18 pages, 8 figures, changes match published version

References in corpus (11)

• Bayes in the sky: Bayesian inference and model selection in cosmology
• Cosmology and Fundamental Physics with the Euclid Satellite
• Planck Early Results: The Planck mission
• Planck Early Results: All sky temperature and dust optical depth from Planck and IRAS: Constraints on the "dark gas" in our galaxy
• Measuring the effective complexity of cosmological models
• COrE (Cosmic Origins Explorer) A White Paper
• A new perspective on Dark Energy modeling via Genetic Algorithms
• Bayesian Calibrated Significance Levels Applied to the Spectral Tilt and Hemispherical Asymmetry
• SN and BAO constraints on (new) polynomial dark energy parametrizations: current results and forecasts
• The power of Bayesian evidence in astronomy
• Should we doubt the cosmological constant?

Cited by in corpus (84)

• Early Dark Energy Can Resolve The Hubble Tension
• Challenges for $\Lambda$CDM: An update
• Beyond $\Lambda$CDM: Problems, solutions, and the road ahead
• The $H_0$ Olympics: A fair ranking of proposed models
• Observational constraints of $f(Q)$ gravity
• Concordance and Discordance in Cosmology
• Observational constraints on cosmological solutions of $f(Q)$ theories
• The Observational Future of Cosmological Scalar-Tensor Theories
• Hubble tension or a transition of the Cepheid SnIa calibrator parameters?
• Unraveling the effective fluid approach for $f(R)$ models in the subhorizon approximation
• Late-transition vs smooth $H(z)$ deformation models for the resolution of the Hubble crisis
• Measurements of $H_0$ in modified gravity theories: The role of lensed quasars in the late-time Universe
• Is cosmic acceleration proven by local cosmological probes?
• Observational constraints on the deceleration parameter in a tilted universe
• Geometrical observational bounds on a fractal horizon holographic dark energy
• Bayesian analysis of the spin distribution of LIGO/Virgo black holes
• Designing Horndeski and the effective fluid approach
• Non-local gravity and comparison with observational datasets. II. Updated results and Bayesian model comparison with $\Lambda$CDM
• Comparison between different methods of model selection in cosmology
• Comparative analysis of model-independent methods for exploring the nature of dark energy
• Cosmological constraints and comparison of viable $f(R)$ models
• Observational Constraint in $f(R,T)$ gravity from the cosmic chronometers and some standard distance measurement parameters
• Quantifying the evidence for the current speed-up of the Universe with low and intermediate-redshift data. A more model-independent approach
• Cosmological perturbations and observational constraints on nonlocal massive gravity
• A Bayesian comparison between $\Lambda$CDM and phenomenologically emergent dark energy models
• Constraining the Baryonic Feedback with Cosmic Shear Using the DES Year-3 Small-Scale Measurements
• Observational constraints on cosmological future singularities
• Model selection applied to reconstructions of the Dark Energy
• Cosmological constraints on the Generalized Uncertainty Principle from modified Friedmann equations
• Cosmological Constraints on Higgs-Dilaton Inflation
• Dark energy reconstruction based on the PADE approximation; an expansion around the $\Lambda$CDM
• Euclid: constraining dark energy coupled to electromagnetism using astrophysical and laboratory data
• Evaporating primordial black holes as varying dark energy
• The look-elsewhere effect from a unified Bayesian and frequentist perspective
• Cosmological constraints on early dark energy from the full shape analysis of eBOSS DR16
• Breaking the Vainshtein screening in clusters of galaxies
• Angular ellipticity correlations in a composite alignment model for elliptical and spiral galaxies and inference from weak lensing
• On Model Selection in Cosmology
• Statistical hierarchy of varying speed of light cosmologies
• Proximal nested sampling for high-dimensional Bayesian model selection
• $f(R)$ gravity modifications: from the action to the data
• Bayesian Model Selection with Future 21cm Observations of The Epoch of Reionisation
• Machine Learning and cosmographic reconstructions of quintessence and the Swampland conjectures
• Holographic and Gravity-Thermodynamic Approaches in Entropic Cosmology: Bayesian Assessment using late-time Data
• Relativistic accretion disk reflection in AGN X-ray spectra at z=0.5--4: a study of four \textit{Chandra} deep fields
• Testing non-local gravity by clusters of galaxies
• Type Ia supernovae data with scalar-tensor gravity
• SDSS-IV MaNGA: Bayesian analysis of the star formation history of low-mass galaxies in the local Universe
• DGP and DGPish cosmologies from $f(Q)$ actions
• Multi-component DHOST analysis in galaxy clusters
• Frequentist tests for Bayesian models
• A GREAT model comparison against the cosmological constant
• Why weak lensing cluster shapes are insensitive to self-interacting dark matter
• Bayesian evidence of the post-Planck curvaton
• Cosmological Constraints on Entropic Cosmology with Matter Creation
• A generalized $\Lambda$CDM model with parameterized Hubble parameter in particle creation, viscous and $f(R)$ model framework
• On The Distribution of Bayesian Evidences
• Cosmological constraints with the Effective Fluid approach for Modified Gravity
• Reference level of the vacuum energy density of the Universe and astrophysical data
• DHOST gravity in Ultra-diffuse galaxies -- Part II: NGC 1052-DF4 and Dragonfly 44
• Tests of Standard Cosmology in Horava Gravity and Bayesian Evidence for a Closed Universe, and the Hubble Tension
• Interpreting Internal Consistency of DES Measurements
• Comparison of piecewise-constant methods for dark energy
• Mass classification of dark matter perturbers of stellar tidal streams
• Bayesian naturalness, simplicity, and testability applied to the $B-L$ MSSM GUT
• Constraining viscous fluid models in $f(Q)$ gravity with data
• Late-Time Anisotropy Sourced by a 2-Form Field Non-Minimally Coupled to Cold Dark Matter
• Comparing Dark Energy models with Hubble versus Growth Rate data
• Grid-based exploration of cosmological parameter space with Snake
• Constraints on Generalized Gravity-Thermodynamic Cosmology from DESI DR2
• DESI forecast for Dark Matter-Neutrino interactions using EFTofLSS
• Is Dark Energy Changing? Probing the Universe's Expansion with present and future astronomical probes
• $\Lambda CDM$ periodic cosmology
• Gaussian discriminators between $\Lambda$CDM and wCDM cosmologies using expansion data
• DESI constraints on two-field quintessence with exponential potentials
• Kilonova and progenitor properties of merger-driven gamma-ray bursts
• Influence of photometric galaxies redshift distribution in BAO estimation
• DHOST gravity in Ultra-diffuse galaxies -- Part I: the case of NGC1052-DF2
• Observational constraints on a modified-gravity model with an exponential function of the curvature using the expansion history, the RSD, and the Pantheon+SH0ES data
• Dark energy model selection with current and future data
• The $H_0$ and $S_8$ tensions necessitate early and late time changes to $\Lambda$CDM
• Bayesian vs Frequentist: Comparing Bayesian model selection with a frequentist approach using the iterative smoothing method
• Bayesian Model Comparison of $R_h=ct$ versus $\Lambda$CDM using HII galaxy Hubble diagram
• Running vacuum model versus $\Lambda$CDM -- a Bayesian analysis