On methods for correcting for the look-elsewhere effect in searches for new physics
arXiv:1602.03765 · doi:10.1088/1748-0221/11/12/P12010
Abstract
The search for new significant peaks over a energy spectrum often involves a statistical multiple hypothesis testing problem. Separate tests of hypothesis are conducted at different locations producing an ensemble of local p-values, the smallest of which is reported as evidence for the new resonance. Unfortunately, controlling the false detection rate (type I error rate) of such procedures may lead to excessively stringent acceptance criteria. In the recent physics literature, two promising statistical tools have been proposed to overcome these limitations. In 2005, a method to "find needles in haystacks" was introduced by Pilla et al. [1], and a second method was later proposed by Gross and Vitells [2] in the context of the "look elsewhere effect" and trial factors. We show that, for relatively small sample sizes, the former leads to an artificial inflation of statistical power that stems from an increase in the false detection rate, whereas the two methods exhibit similar performance for large sample sizes. We apply the methods to realistic simulations of the Fermi Large Area Telescope data, in particular the search for dark matter annihilation lines. Further, we discuss the counter-intutive scenario where the look-elsewhere corrections are more conservative than much more computationally efficient corrections for multiple hypothesis testing. Finally, we provide general guidelines for navigating the tradeoffs between statistical and computational efficiency when selecting a statistical procedure for signal detection.
References in corpus (14)
- Observation of a new particle in the search for the Standard Model Higgs boson with the ATLAS detector at the LHC
- Observation of a new boson at a mass of 125 GeV with the CMS experiment at the LHC
- The Large Area Telescope on the Fermi Gamma-ray Space Telescope Mission
- A Unified Approach to the Classical Statistical Analysis of Small Signals
- Trial factors for the look elsewhere effect in high energy physics
- A Tentative Gamma-Ray Line from Dark Matter Annihilation at the Fermi Large Area Telescope
- Updated Search for Spectral Lines from Galactic Dark Matter Interactions with Pass 8 Data from the Fermi Large Area Telescope
- The flattening of the concentration-mass relation towards low halo masses and its implications for the annihilation signal boost
- On Computing Upper Limits to Source Intensities
- Search for Gamma-Ray Lines towards Galaxy Clusters with the Fermi-LAT
- Estimating the significance of a signal in a multi-dimensional search
- Statistical Issues in Astrophysical Searches for Particle Dark Matter
- A method for comparing non-nested models with application to astrophysical searches for new physics
- A New Technique for Finding Needles in Haystacks: A Geometric Approach to Distinguishing Between a New Source and Random Fluctuations
Cited by in corpus (14)
- Search for axion-like dark matter through nuclear spin precession in electric and magnetic fields
- Search for axionlike dark matter with a liquid-state nuclear spin comagnetometer
- Multi-messenger characterization of Mrk 501 during historically low X-ray and -ray activity
- Rapid X-ray Variability in Mkn 421 during a Multiwavelength Campaign
- Testing One Hypothesis Multiple Times: The Multidimensional Case
- A survey for variable young stars with small telescopes: IV -- Rotation Periods of YSOs in IC5070
- Detecting new signals under background mismodelling
- Search for galactic Pevatron candidates in a population of unidentified gamma-ray sources
- Self-Calibrating the Look-Elsewhere Effect: Fast Evaluation of the Statistical Significance Using Peak Heights
- Nested sampling for frequentist computation: fast estimation of small -values
- Observation of exotic resonant structure in diffractive processes in proton-proton collisions
- Sequential hypothesis testing for Axion Haloscopes
- A novel approach to detect line emission under high background in high-resolution X-ray spectra
- Testing the correlation between bending angle and polarization properties of bent radio galaxies