On hypothesis testing, trials factor, hypertests and the BumpHunter

Abstract

A detailed presentation of hypothesis testing is given. The "look elsewhere" effect is illustrated, and a treatment of the trials factor is proposed with the introduction of hypothesis hypertests. An example of such a hypertest is presented, named BumpHunter, which is used in the recent ATLAS dijet resonance search, and in an earlier version in the CDF Global Search, to look for exotic phenomena in high energy physics. As a demonstration, the BumpHunter is used to address Problem 1 of the Banff Challenge.

34 pages, 91 figures, prepared for PHYSTAT2011

References in corpus (4)

• The RooStats Project
• Global Search for New Physics with 2.0/fb at CDF
• Model-Independent and Quasi-Model-Independent Search for New Physics at CDF
• A general search for new phenomena in ep scattering at HERA

Cited by in corpus (44)

• Search for new phenomena in the dijet mass distribution using $p-p$ collision data at $\sqrt{s}=8$ TeV with the ATLAS detector
• Search for new high-mass phenomena in the dilepton final state using 36 fb${}^{-1}$ of proton-proton collision data at $\sqrt{s}$ = 13 TeV with the ATLAS detector
• Anomaly Detection for Resonant New Physics with Machine Learning
• Learning New Physics from a Machine
• Search for new phenomena in dijet events using 37 fb${}^{-1}$ of $pp$ collision data collected at $\sqrt{s}=$13 TeV with the ATLAS detector
• Search for New Phenomena in Dijet Mass and Angular Distributions from $pp$ Collisions at $\sqrt{s}$ = 13 TeV with the ATLAS Detector
• Search for new resonances in mass distributions of jet pairs using 139 fb${}^{-1}$ of $pp$ collisions at $\sqrt{s}=13$ TeV with the ATLAS detector
• Search for New Physics in Dijet Mass and Angular Distributions in pp Collisions at sqrt(s) = 7 TeV Measured with the ATLAS Detector
• Search for high-mass diphoton resonances in $pp$ collisions at $\sqrt{s}=8$ TeV with the ATLAS detector
• A search for $t\bar{t}$ resonances using lepton-plus-jets events in proton-proton collisions at $\sqrt{s}=8$ TeV with the ATLAS detector
• Guiding New Physics Searches with Unsupervised Learning
• ATLAS search for new phenomena in dijet mass and angular distributions using pp collisions at sqrt(s)=7 TeV
• The Dark Machines Anomaly Score Challenge: Benchmark Data and Model Independent Event Classification for the Large Hadron Collider
• Search for a high-mass Higgs boson decaying to a $W$ boson pair in $pp$ collisions at $\sqrt{s}=8$ TeV with the ATLAS detector
• Search for low-mass dijet resonances using trigger-level jets with the ATLAS detector in $pp$ collisions at $\sqrt{s}=13$ TeV
• Search for heavy particles decaying into top-quark pairs using lepton-plus-jets events in proton-proton collisions at $\sqrt{s}=13$ TeV with the ATLAS detector
• Search for high-mass new phenomena in the dilepton final state using proton-proton collisions at $\sqrt{s}=13$ TeV with the ATLAS detector
• A search for ttbar resonances in lepton+jets events with highly boosted top quarks collected in pp collisions at sqrt(s) = 7 TeV with the ATLAS detector
• Search for Extra Dimensions in diphoton events using proton-proton collisions recorded at sqrt(s) = 7 TeV with the ATLAS detector at the LHC
• Search for heavy particles decaying into a top-quark pair in the fully hadronic final state in $pp$ collisions at $\sqrt{s}=13$ TeV with the ATLAS detector
• Search for new phenomena in photon+jet events collected in proton--proton collisions at sqrt(s) = 8 TeV with the ATLAS detector
• Search for low-mass resonances decaying into two jets and produced in association with a photon using $pp$ collisions at $\sqrt{s}=13$ TeV with the ATLAS detector
• Search for resonances in the mass distribution of jet pairs with one or two jets identified as $b$-jets in proton--proton collisions at $\sqrt{s}=13$ TeV with the ATLAS detector
• RPV stops bump off the background
• The LHC Olympics 2020: A Community Challenge for Anomaly Detection in High Energy Physics
• Search for $t\bar{t}$ resonances in fully hadronic final states in $pp$ collisions at $\sqrt{s}=13$ TeV with the ATLAS detector
• Search for diphoton events with large missing transverse momentum in 7 TeV proton-proton collision data with the ATLAS detector
• Search for the production of single vector-like and excited quarks in the $Wt$ final state in $pp$ collisions at $\sqrt{s}$ = 8 TeV with the ATLAS detector
• Search for lepton-flavor violation in different-flavor, high-mass final states in $pp$ collisions at $\sqrt{s}=13$ TeV with the ATLAS detector
• Top Partner Discovery in the $T\to tZ$ channel at the LHC
• Fully Bayesian Unfolding
• A search for ttbar resonances with the ATLAS detector in 2.05 fb^-1 of proton-proton collisions at sqrt(s) = 7 TeV
• Search for new particles decaying to ZZ using final states with leptons and jets with the ATLAS detector in sqrt(s) = 7 Tev proton-proton collisions
• Search for dijet resonances in events with an isolated charged lepton using $\sqrt{s}=13$ TeV proton-proton collision data collected by the ATLAS detector
• Ephemeral Learning -- Augmenting Triggers with Online-Trained Normalizing Flows
• Search for low-scale gravity signatures in multi-jet final states with the ATLAS detector at $\sqrt{s}=8$ TeV
• Pursuit of paired dijet resonances in the Run 2 dataset with ATLAS
• pyBumpHunter: A model independent bump hunting tool in Python for High Energy Physics analyses
• Hunting for bumps in the margins
• Testing One Hypothesis Multiple times
• Particle Graph Autoencoders and Differentiable, Learned Energy Mover's Distance
• Dijet Searches for New Physics in ATLAS
• Search for New Physics Involving Top Quarks at ATLAS
• LHC searches for physics beyond the Standard Model with top quarks