Simple and statistically sound recommendations for analysing physical theories

Abstract

Physical theories that depend on many parameters or are tested against data from many different experiments pose unique challenges to statistical inference. Many models in particle physics, astrophysics and cosmology fall into one or both of these categories. These issues are often sidestepped with statistically unsound ad hoc methods, involving intersection of parameter intervals estimated by multiple experiments, and random or grid sampling of model parameters. Whilst these methods are easy to apply, they exhibit pathologies even in low-dimensional parameter spaces, and quickly become problematic to use and interpret in higher dimensions. In this article we give clear guidance for going beyond these procedures, suggesting where possible simple methods for performing statistically sound inference, and recommendations of readily-available software tools and standards that can assist in doing so. Our aim is to provide any physicists lacking comprehensive statistical training with recommendations for reaching correct scientific conclusions, with only a modest increase in analysis burden. Our examples can be reproduced with the code publicly available at https://doi.org/10.5281/zenodo.4322283.

15 pages, 4 figures. extended discussions. closely matches version accepted for publication

References in corpus (13)

• Observation of a new particle in the search for the Standard Model Higgs boson with the ATLAS detector at the LHC
• Observation of a new boson at a mass of 125 GeV with the CMS experiment at the LHC
• Bayes in the sky: Bayesian inference and model selection in cosmology
• The global electroweak fit at NNLO and prospects for the LHC and ILC
• PolyChord: nested sampling for cosmology
• Natural Priors, CMSSM Fits and LHC Weather Forecasts
• Fundamental statistical limitations of future dark matter direct detection experiments
• Use of event-level neutrino telescope data in global fits for theories of new physics
• A Coverage Study of the CMSSM Based on ATLAS Sensitivity Using Fast Neural Networks Techniques
• A comparison of optimisation algorithms for high-dimensional particle and astrophysics applications
• Reproducibility and Replication of Experimental Particle Physics Results
• Statistical coverage for supersymmetric parameter estimation: a case study with direct detection of dark matter
• Comment on "Bayesian Analysis of Pentaquark Signals from CLAS Data", with Response to the Reply by Ireland and Protopopsecu

Cited by in corpus (13)

• Cosmological phase transitions: from perturbative particle physics to gravitational waves
• Nested sampling for physical scientists
• Low energy SUSY confronted with new measurements of W-boson mass and muon g-2
• $B$ meson anomalies and large $B^{+}\to K^{+}\nu \bar{\nu }$ in non-universal $U(1{)}^{\prime }$ models
• Hunting WIMPs with LISA: Correlating dark matter and gravitational wave signals
• Active learning BSM parameter spaces
• Exploring phase space with Nested Sampling
• Global fits of SUSY at future Higgs factories
• PEANUTS: a software for the automatic computation of solar neutrino flux and its propagation within Earth
• Relaxing Limits from Big Bang Nucleosynthesis on Heavy Neutral Leptons with Axion-like Particles
• A Global Fit of Non-Relativistic Effective Dark Matter Operators Including Solar Neutrinos
• Explaining the hints for lepton flavour universality violation with three $S_2$ leptoquark generations
• Bring the noise: exact inference from noisy simulations in collider physics