An importance sampling method for Feldman-Cousins confidence intervals

Abstract

In various high-energy physics contexts, such as neutrino-oscillation experiments, several assumptions underlying the typical asymptotic confidence interval construction are violated, such that one has to resort to computationally expensive methods like the Feldman-Cousins method for obtaining confidence intervals with proper statistical coverage. By construction, the computation of intervals at high confidence levels requires fitting millions or billions of pseudo-experiments, while wasting most of the computational cost on overly precise intervals at low confidence levels. In this work, a simple importance sampling method is introduced which reuses pseudo-experiments produced for all tested parameter values in a single mixture distribution. This results in a significant error reduction on the estimated critical values, especially at high confidence levels, and simultaneously yields a correct interpolation of these critical values between the parameter values at which the pseudo-experiments were produced. The theoretically calculated performance is demonstrated numerically using a simple example from the analysis of neutrino oscillations. The relationship to similar techniques applied in statistical mechanics and $p$-value computations is discussed.

arXiv comment16 pages, 7 figures

References in corpus (6)

• Observation of a new particle in the search for the Standard Model Higgs boson with the ATLAS detector at the LHC
• A Unified Approach to the Classical Statistical Analysis of Small Signals
• The T2K Experiment
• Practical Statistics for the LHC
• Gaussian Process Accelerated Feldman-Cousins Approach for Physical Parameter Inference
• Nested sampling for frequentist computation: fast estimation of small $p$-values

Cited by in corpus (1)

• Monte Carlo method for constructing confidence intervals with unconstrained and constrained nuisance parameters in the NOvA experiment