Another Look at Confidence Intervals: Proposal for a More Relevant and Transparent Approach

Abstract

The behaviors of various confidence/credible interval constructions are explored, particularly in the region of low statistics where methods diverge most. We highlight a number of challenges, such as the treatment of nuisance parameters, and common misconceptions associated with such constructions. An informal survey of the literature suggests that confidence intervals are not always defined in relevant ways and are too often misinterpreted and/or misapplied. This can lead to seemingly paradoxical behaviours and flawed comparisons regarding the relevance of experimental results. We therefore conclude that there is a need for a more pragmatic strategy which recognizes that, while it is critical to objectively convey the information content of the data, there is also a strong desire to derive bounds on models and a natural instinct to interpret things this way. Accordingly, we attempt to put aside philosophical biases in favor of a practical view to propose a more transparent and self-consistent approach that better addresses these issues.

arXiv comment23 pages, 11 figures

References in corpus (7)

• Search for the Standard Model Higgs Boson at LEP
• First observation of PeV-energy neutrinos with IceCube
• Limits and Confidence Intervals in the Presence of Nuisance Parameters
• Search for Neutrinoless Double-Beta Decay in ${}^{136}$Xe with EXO-200
• Independent Measurement of the Total Active 8B Solar Neutrino Flux Using an Array of 3He Proportional Counters at the Sudbury Neutrino Observatory
• Latest Results of the KARMEN2 Experiment
• The Search for Neutrino Oscillations numubar->nuebar with KARMEN

Cited by in corpus (7)

• The IceCube high-energy starting event sample: Description and flux characterization with 7.5 years of data
• Background free search for neutrinoless double beta decay with GERDA Phase II
• A Global Analysis of Dark Matter Signals from 27 Dwarf Spheroidal Galaxies using 11 Years of Fermi-LAT Observations
• Analysis methods for the first KATRIN neutrino-mass measurement
• Statistical Issues in Astrophysical Searches for Particle Dark Matter
• Combined Constraints on Majorana Masses from Neutrinoless Double Beta Decay Experiments
• Searching for high-energy gamma-ray counterparts to Gravitational Wave sources with Fermi-LAT: a needle in a haystack