Self-Calibrating the Look-Elsewhere Effect: Fast Evaluation of the Statistical Significance Using Peak Heights

Abstract

In experiments where one searches a large parameter space for an anomaly, one often finds many spurious noise-induced peaks in the likelihood. This is known as the look-elsewhere effect, and must be corrected for when performing statistical analysis. This paper introduces a method to calibrate the false alarm probability (FAP), or $p$-value, for a given dataset by considering the heights of the highest peaks in the likelihood. In the simplest form of self-calibration, the look-elsewhere-corrected ${\chi }^2$ of a physical peak is approximated by the ${\chi }^2$ of the peak minus the ${\chi }^2$ of the highest noise-induced peak. Generalizing this concept to consider lower peaks provides a fast method to quantify the statistical significance with improved accuracy. In contrast to alternative methods, this approach has negligible computational cost as peaks in the likelihood are a byproduct of every peak-search analysis. We apply to examples from astronomy, including planet detection, periodograms, and cosmology.

12 pages, 7 figures

References in corpus (8)

• Observation of a new particle in the search for the Standard Model Higgs boson with the ATLAS detector at the LHC
• Observation of a new boson at a mass of 125 GeV with the CMS experiment at the LHC
• Assessing statistical significance of periodogram peaks
• Combining power spectrum and bispectrum measurements to detect oscillatory features
• Keplerian periodogram for Doppler exoplanets detection: optimized computation and analytic significance thresholds
• Detecting non-sinusoidal periodicities in observational data using multi-harmonic periodograms
• Periodogram and likelihood periodicity search in the SNO solar neutrino data
• Likelihood-Ratio Ranking Statistic for Compact Binary Coalescence Candidates with Rate Estimation