p-Values for Model Evaluation

Abstract

Deciding whether a model provides a good description of data is often based on a goodness-of-fit criterion summarized by a p-value. Although there is considerable confusion concerning the meaning of p-values, leading to their misuse, they are nevertheless of practical importance in common data analysis tasks. We motivate their application using a Bayesian argumentation. We then describe commonly and less commonly known discrepancy variables and how they are used to define p-values. The distribution of these are then extracted for examples modeled on typical data analysis tasks, and comments on their usefulness for determining goodness-of-fit are given.

33 pages, 13 figures; added figure for runs test, changed to coherent notation, updated bibliography; fixed formula in the appendix

References in corpus (2)

• Signal discovery in sparse spectra: a Bayesian analysis
• A Test Statistic for Weighted Runs

Cited by in corpus (13)

• Results from 730 kg days of the CRESST-II Dark Matter Search
• Measurement of the half-life of the two-neutrino double beta decay of Ge-76 with the Gerda experiment
• Search for Two-Neutrino Double Electron Capture of ${}^{124}$Xe with XENON100
• Statistical Methods Applied to the Search of Sterile Neutrinos
• Modeling of GERDA Phase II data
• New Limits on Double Electron Capture of ${}^{40}$Ca and ${}^{180}$W
• GeMSE: a Low-Background Facility for Gamma-Spectrometry at Moderate Rock Overburden
• Study of proton parton distribution functions at high x using ZEUS data
• BFKL Evolution as a Communicator Between Small and Large Energy Scales
• Is the bump significant? An axion-search example
• The CRESST II Dark Matter Search
• PartonDensity.jl: a novel parton density determination code
• The GeMSE Facility for Low-Background γ-Ray Spectrometry