ABCDisCo: Automating the ABCD Method with Machine Learning

Abstract

The ABCD method is one of the most widely used data-driven background estimation techniques in high energy physics. Cuts on two statistically-independent classifiers separate signal and background into four regions, so that background in the signal region can be estimated simply using the other three control regions. Typically, the independent classifiers are chosen "by hand" to be intuitive and physically motivated variables. Here, we explore the possibility of automating the design of one or both of these classifiers using machine learning. We show how to use state-of-the-art decorrelation methods to construct powerful yet independent discriminators. Along the way, we uncover a previously unappreciated aspect of the ABCD method: its accuracy hinges on having low signal contamination in control regions not just overall, but relative to the signal fraction in the signal region. We demonstrate the method with three examples: a simple model consisting of three-dimensional Gaussians; boosted hadronic top jet tagging; and a recasted search for paired dijet resonances. In all cases, automating the ABCD method with machine learning significantly improves performance in terms of ABCD closure, background rejection and signal contamination.

37 pages, 12 figures

References in corpus (16)

• PYTHIA 6.4 Physics and Manual
• An Introduction to PYTHIA 8.2
• Measuring and testing dependence by correlation of distances
• Brownian distance covariance
• Top-tagging: A Method for Identifying Boosted Hadronic Tops
• Extending the Bump Hunt with Machine Learning
• Jet Flavor Classification in High-Energy Physics with Deep Neural Networks
• A generic anti-QCD jet tagger
• Parton Shower Uncertainties in Jet Substructure Analyses with Deep Neural Networks
• Search for pair-produced resonances decaying to jet pairs in proton-proton collisions at sqrt(s) = 8 TeV
• A Living Review of Machine Learning for Particle Physics
• Variational Autoencoders for Anomalous Jet Tagging
• Transferability of Deep Learning Models in Searches for New Physics at Colliders
• Search for new phenomena in final states with large jet multiplicities and missing transverse momentum using $\sqrt{(}s)=13$ TeV proton$-$proton collisions recorded by ATLAS in Run 2 of the LHC
• Towards Machine Learning Analytics for Jet Substructure
• New approaches for boosting to uniformity

Cited by in corpus (32)

• Simulation-Assisted Decorrelation for Resonant Anomaly Detection
• Modern Machine Learning and Particle Physics
• Machine Learning for Anomaly Detection in Particle Physics
• Uncertainty Aware Learning for High Energy Physics
• Quasi Anomalous Knowledge: Searching for new physics with embedded knowledge
• Online-compatible Unsupervised Non-resonant Anomaly Detection
• Symmetries, Safety, and Self-Supervision
• Search for displaced leptons in $\sqrt{s}=13$ TeV $pp$ collisions with the ATLAS detector
• A Cautionary Tale of Decorrelating Theory Uncertainties
• Search for supersymmetry in final states with disappearing tracks in proton-proton collisions at $\sqrt{s}$ = 13 TeV
• Search for pair-produced vector-like leptons in final states with third-generation leptons and at least three b quark jets in proton-proton collisions at $\sqrt{s}$ = 13 TeV
• Enhancing searches for resonances with machine learning and moment decomposition
• Search for long-lived particles decaying in the CMS muon detectors in proton-proton collisions at $\sqrt{s}$ = 13 TeV
• Data-driven Estimation of Background Distribution through Neural Autoregressive Flows
• Search for long-lived particles using displaced vertices and missing transverse momentum in proton-proton collisions at $\sqrt{s}$ = 13 TeV
• Bias and Priors in Machine Learning Calibrations for High Energy Physics
• Bayesian Probabilistic Modelling for Four-Tops at the LHC
• Search for neutral long-lived particles that decay into displaced jets in the ATLAS calorimeter in association with leptons or jets using $pp$ collisions at $\sqrt{s}=13$ TeV
• Non-resonant Anomaly Detection with Background Extrapolation
• Measurement of the $Z$ boson production cross-section in proton-lead collisions at $\sqrt{s_{\mathrm{NN}}}=8.16\mathrm{TeV}$
• Unsupervised and lightly supervised learning in particle physics
• Invariant Representation Driven Neural Classifier for Anti-QCD Jet Tagging
• Improvement and generalization of ABCD method with Bayesian inference
• Search for pairs of muons with small displacements in $pp$ collisions at $\sqrt{s}=13$ TeV with the ATLAS detector
• Designing Observables for Measurements with Deep Learning
• Data-driven method to estimate contamination from light ion beam transmutation at colliders
• Search for pair-produced higgsinos decaying via Higgs or $Z$ bosons to final states containing a pair of photons and a pair of $b$-jets with the ATLAS detector
• Search for b hadron decays to long-lived particles in the CMS endcap muon detectors
• Solving Combinatorial Problems at Particle Colliders Using Machine Learning
• Data driven background estimation in HEP using Generative Adversarial Networks
• Inferring correlated distributions: boosted top jets
• Machine learning in top quark physics at ATLAS and CMS