Nested sampling for frequentist computation: fast estimation of small $p$-values

Abstract

We propose a novel method for computing $p$-values based on nested sampling (NS) applied to the sampling space rather than the parameter space of the problem, in contrast to its usage in Bayesian computation. The computational cost of NS scales as ${\log }^{2}1/p$, which compares favorably to the $1/p$ scaling for Monte Carlo (MC) simulations. For significances greater than about $4\sigma$ in both a toy problem and a simplified resonance search, we show that NS requires orders of magnitude fewer simulations than ordinary MC estimates. This is particularly relevant for high-energy physics, which adopts a $5\sigma$ gold standard for discovery. We conclude with remarks on new connections between Bayesian and frequentist computation and possibilities for tuning NS implementations for still better performance in this setting.

arXiv comment5 pages + refs, 3 figures. added refs + schematic of algorithm. closely matches published version

References in corpus (26)

• Observation of a new particle in the search for the Standard Model Higgs boson with the ATLAS detector at the LHC
• Observation of a new boson at a mass of 125 GeV with the CMS experiment at the LHC
• Asymptotic formulae for likelihood-based tests of new physics
• Multimodal nested sampling: an efficient and robust alternative to MCMC methods for astronomical data analysis
• Importance Nested Sampling and the MultiNest Algorithm
• The global electroweak fit at NNLO and prospects for the LHC and ILC
• PolyChord: nested sampling for cosmology
• Excess Electronic Recoil Events in XENON1T
• Searching for Dark Matter Annihilation in Recently Discovered Milky Way Satellites with Fermi-LAT
• Trial factors for the look elsewhere effect in high energy physics
• Study of the mass and spin-parity of the Higgs boson candidate via its decays to Z boson pairs
• Quantifying the sensitivity of oscillation experiments to the neutrino mass ordering
• anesthetic: nested sampling visualisation
• Killing the cMSSM softly
• Searching for new physics with profile likelihoods: Wilks and beyond
• A strategy for a general search for new phenomena using data-derived signal regions and its application within the ATLAS experiment
• Recursive Pathways to Marginal Likelihood Estimation with Prior-Sensitivity Analysis
• On the significance of new physics in $b\to s{\ell }^{+}{\ell }^{-}$ decays
• Global fits of axion-like particles to XENON1T and astrophysical data
• MUSiC: a model-unspecific search for new physics in proton-proton collisions at $\sqrt{s}=$ 13 TeV
• Search for supersymmetry with razor variables in pp collisions at sqrt(s) = 7 TeV
• Reproducibility and Replication of Experimental Particle Physics Results
• Statistical interpretation of sterile neutrino oscillation searches at reactors
• On methods for correcting for the look-elsewhere effect in searches for new physics
• Nested sampling with plateaus
• Numerical Computation of p-values with myFitter

Cited by in corpus (4)

• Nested sampling for physical scientists
• A comparison of Bayesian sampling algorithms for high-dimensional particle physics and cosmology applications
• Exploring phase space with Nested Sampling
• An importance sampling method for Feldman-Cousins confidence intervals