Self-Calibrating the Look-Elsewhere Effect: Fast Evaluation of the Statistical Significance Using Peak Heights

Abstract

In experiments where one searches a large parameter space for an anomaly, one often finds many spurious noise-induced peaks in the likelihood. This is known as the look-elsewhere effect, and must be corrected for when performing statistical analysis. This paper introduces a method to calibrate the false alarm probability (FAP), or $p$-value, for a given dataset by considering the heights of the highest peaks in the likelihood. In the simplest form of self-calibration, the look-elsewhere-corrected ${\chi }^2$ of a physical peak is approximated by the ${\chi }^2$ of the peak minus the ${\chi }^2$ of the highest noise-induced peak. Generalizing this concept to consider lower peaks provides a fast method to quantify the statistical significance with improved accuracy. In contrast to alternative methods, this approach has negligible computational cost as peaks in the likelihood are a byproduct of every peak-search analysis. We apply to examples from astronomy, including planet detection, periodograms, and cosmology.

arXiv comment12 pages, 7 figures

References in corpus (37)

• Observation of a new particle in the search for the Standard Model Higgs boson with the ATLAS detector at the LHC
• Observation of a new boson at a mass of 125 GeV with the CMS experiment at the LHC
• Efficient Computation of CMB anisotropies in closed FRW models
• Cosmological parameters from CMB and other data: a Monte-Carlo approach
• Asymptotic formulae for likelihood-based tests of new physics
• Planck 2013 results. I. Overview of products and scientific results
• Understanding the Lomb-Scargle Periodogram
• Kepler Mission Design, Realized Photometric Performance, and Early Science
• Observation of High-Energy Astrophysical Neutrinos in Three Years of IceCube Data
• Planck 2013 results. XV. CMB power spectra and likelihood
• GW150914: First results from the search for binary black hole coalescence with Advanced LIGO
• Trial factors for the look elsewhere effect in high energy physics
• Analysis Framework for the Prompt Discovery of Compact Binary Mergers in Gravitational-wave Data
• Assessing statistical significance of periodogram peaks
• A Precision Search for WIMPs with Charged Cosmic Rays
• Planes of satellite galaxies: when exceptions are the rule
• Combining power spectrum and bispectrum measurements to detect oscillatory features
• Integration of inhomogeneous cosmological spacetimes in the BSSN formalism
• Polyspectra searches for sharp oscillatory features in cosmic microwave sky data
• Search for Gamma-Ray Lines towards Galaxy Clusters with the Fermi-LAT
• Search for features in the spectrum of primordial perturbations using Planck and other datasets
• A Search for a keV Signature of Radiatively Decaying Dark Matter with Suzaku XIS Observations of the X-ray Diffuse Background
• Efficient modeling of correlated noise I. Statistical significance of periodogram peaks
• Detecting non-sinusoidal periodicities in observational data: the von Mises periodogram for variable stars and exoplanetary transits
• Do high energy astrophysical neutrinos trace star formation?
• The look-elsewhere effect from a unified Bayesian and frequentist perspective
• Keplerian periodogram for Doppler exoplanets detection: optimized computation and analytic significance thresholds
• Periodogram and likelihood periodicity search in the SNO solar neutrino data
• Detecting non-sinusoidal periodicities in observational data using multi-harmonic periodograms
• On methods for correcting for the look-elsewhere effect in searches for new physics
• Matched filtering with non-Gaussian noise for planet transit detections
• Statistical detection of patterns in unidimensional distributions by continuous wavelet transforms
• Kepler data analysis: non-Gaussian noise and Fourier Gaussian process analysis of star variability
• Likelihood-Ratio Ranking Statistic for Compact Binary Coalescence Candidates with Rate Estimation
• Analysing the Main Belt asteroid distributions by wavelets
• Fine-resolution analysis of exoplanetary distributions by wavelets: hints of an overshooting iceline accumulation
• Massive search of spot- and facula-crossing events in 1598 exoplanetary transit lightcurves

Cited by in corpus (2)

• An Independent Search for Small Long-period Planets in Kepler Data I: Detection Pipeline
• Fast estimation of the look-elsewhere effect using Gaussian random fields