On methods for correcting for the look-elsewhere effect in searches for new physics

Abstract

The search for new significant peaks over a energy spectrum often involves a statistical multiple hypothesis testing problem. Separate tests of hypothesis are conducted at different locations producing an ensemble of local p-values, the smallest of which is reported as evidence for the new resonance. Unfortunately, controlling the false detection rate (type I error rate) of such procedures may lead to excessively stringent acceptance criteria. In the recent physics literature, two promising statistical tools have been proposed to overcome these limitations. In 2005, a method to "find needles in haystacks" was introduced by Pilla et al. [1], and a second method was later proposed by Gross and Vitells [2] in the context of the "look elsewhere effect" and trial factors. We show that, for relatively small sample sizes, the former leads to an artificial inflation of statistical power that stems from an increase in the false detection rate, whereas the two methods exhibit similar performance for large sample sizes. We apply the methods to realistic simulations of the Fermi Large Area Telescope data, in particular the search for dark matter annihilation lines. Further, we discuss the counter-intutive scenario where the look-elsewhere corrections are more conservative than much more computationally efficient corrections for multiple hypothesis testing. Finally, we provide general guidelines for navigating the tradeoffs between statistical and computational efficiency when selecting a statistical procedure for signal detection.

References in corpus (7)

• Observation of a new particle in the search for the Standard Model Higgs boson with the ATLAS detector at the LHC
• Observation of a new boson at a mass of 125 GeV with the CMS experiment at the LHC
• A Tentative Gamma-Ray Line from Dark Matter Annihilation at the Fermi Large Area Telescope
• Search for Gamma-Ray Lines towards Galaxy Clusters with the Fermi-LAT
• Statistical Issues in Astrophysical Searches for Particle Dark Matter
• A method for comparing non-nested models with application to astrophysical searches for new physics
• A New Technique for Finding Needles in Haystacks: A Geometric Approach to Distinguishing Between a New Source and Random Fluctuations

Cited by in corpus (14)

• Search for axion-like dark matter through nuclear spin precession in electric and magnetic fields
• Search for axionlike dark matter with a liquid-state nuclear spin comagnetometer
• Multi-messenger characterization of Mrk 501 during historically low X-ray and $\gamma$-ray activity
• Rapid X-ray Variability in Mkn 421 during a Multiwavelength Campaign
• Testing One Hypothesis Multiple Times: The Multidimensional Case
• A survey for variable young stars with small telescopes: IV -- Rotation Periods of YSOs in IC5070
• Detecting new signals under background mismodelling
• Search for galactic Pevatron candidates in a population of unidentified gamma-ray sources
• Self-Calibrating the Look-Elsewhere Effect: Fast Evaluation of the Statistical Significance Using Peak Heights
• Nested sampling for frequentist computation: fast estimation of small $p$-values
• Observation of exotic $J/\psi \varphi$ resonant structure in diffractive processes in proton-proton collisions
• Sequential hypothesis testing for Axion Haloscopes
• A novel approach to detect line emission under high background in high-resolution X-ray spectra
• Testing the correlation between bending angle and polarization properties of bent radio galaxies