The Jeffreys-Lindley Paradox and Discovery Criteria in High Energy Physics

Abstract

The Jeffreys-Lindley paradox displays how the use of a p-value (or number of standard deviations z) in a frequentist hypothesis test can lead to an inference that is radically different from that of a Bayesian hypothesis test in the form advocated by Harold Jeffreys in the 1930s and common today. The setting is the test of a well-specified null hypothesis (such as the Standard Model of elementary particle physics, possibly with "nuisance parameters") versus a composite alternative (such as the Standard Model plus a new force of nature of unknown strength). The p-value, as well as the ratio of the likelihood under the null hypothesis to the maximized likelihood under the alternative, can strongly disfavor the null hypothesis, while the Bayesian posterior probability for the null hypothesis can be arbitrarily large. The academic statistics literature contains many impassioned comments on this paradox, yet there is no consensus either on its relevance to scientific communication or on its correct resolution. The paradox is quite relevant to frontier research in high energy physics. This paper is an attempt to explain the situation to both physicists and statisticians, in the hope that further progress can be made.

v4: Continued editing for clarity. Figure added. v5: Minor fixes to biblio. Same as published version except for minor copy-edits, Synthese (2014). v6: fix typos, and restore garbled sentence at beginning of Sec 4 to v3

References in corpus (8)

• Observation of a new particle in the search for the Standard Model Higgs boson with the ATLAS detector at the LHC
• Observation of a new boson at a mass of 125 GeV with the CMS experiment at the LHC
• Measurement of the properties of a Higgs boson in the four-lepton final state
• Measurements of Higgs boson production and couplings in diboson final states with the ATLAS detector at the LHC
• Study of the mass and spin-parity of the Higgs boson candidate via its decays to Z boson pairs
• Measurement of the B(s) to mu+ mu- branching fraction and search for B0 to mu+ mu- with the CMS Experiment
• Criteria for Bayesian model choice with application to variable selection
• Measurement of the $B_s^0\to {\mu }^{+}{\mu }^{-}$ branching fraction and search for $B^0\to {\mu }^{+}{\mu }^{-}$ decays at the LHCb experiment

Cited by in corpus (13)

• Semantic and Cognitive Tools to Aid Statistical Science: Replace Confidence and Significance by Compatibility and Surprise
• Bayesian model averaging for analysis of lattice field theory results
• Comparison between different methods of model selection in cosmology
• Improved information criteria for Bayesian model averaging in lattice field theory
• Connecting Simple and Precise P-values to Complex and Ambiguous Realities
• Umbrella sampling: a powerful method to sample tails of distributions
• Bayesian inference with information content model check for Langevin equations
• Simple and statistically sound recommendations for analysing physical theories
• On the correspondence between thermodynamics and inference
• On new physics searches with multidimensional differential shapes
• Bayesian and frequentist approaches to resonance searches
• Reporting Results in High Energy Physics Publications: a Manifesto
• Objective Bayesian approach to the Jeffreys-Lindley paradox