The Dark Machines Anomaly Score Challenge: Benchmark Data and Model Independent Event Classification for the Large Hadron Collider

Abstract

We describe the outcome of a data challenge conducted as part of the Dark Machines Initiative and the Les Houches 2019 workshop on Physics at TeV colliders. The challenged aims at detecting signals of new physics at the LHC using unsupervised machine learning algorithms. First, we propose how an anomaly score could be implemented to define model-independent signal regions in LHC searches. We define and describe a large benchmark dataset, consisting of >1 Billion simulated LHC events corresponding to $10{\mathrm{fb}}^{-1}$ of proton-proton collisions at a center-of-mass energy of 13 TeV. We then review a wide range of anomaly detection and density estimation algorithms, developed in the context of the data challenge, and we measure their performance in a set of realistic analysis environments. We draw a number of useful conclusions that will aid the development of unsupervised new physics searches during the third run of the LHC, and provide our benchmark dataset for future studies at https://www.phenoMLdata.org. Code to reproduce the analysis is provided at https://github.com/bostdiek/DarkMachines-UnsupervisedChallenge.

v1: 54 pages, 24 figures. v2: 56 pages, citations added, extend discussion of look-elsewhere-effect, results unchanged; v3. minor typos and updated references

References in corpus (65)

• Adam: A Method for Stochastic Optimization
• Scikit-learn: Machine Learning in Python
• Auto-Encoding Variational Bayes
• The automated computation of tree-level and next-to-leading order differential cross sections, and their matching to parton shower simulations
• An Introduction to PYTHIA 8.2
• FastJet user manual
• Generative Adversarial Networks
• Asymptotic formulae for likelihood-based tests of new physics
• DELPHES 3, A modular framework for fast simulation of a generic collider experiment
• ALPGEN, a generator for hard multiparton processes in hadronic collisions
• LHAPDF6: parton density access in the LHC precision era
• Parton distributions from high-precision collider data
• Variational Inference with Normalizing Flows
• R-parity violating supersymmetry
• Trial factors for the look elsewhere effect in high energy physics
• MADE: Masked Autoencoder for Distribution Estimation
• Anomaly Detection with Density Estimation
• Classification without labels: Learning from mixed samples in high energy physics
• Searching for New Physics with Deep Autoencoders
• Anomaly Detection for Resonant New Physics with Machine Learning
• Energy Flow Networks: Deep Sets for Particle Jets
• Phenomenology of the minimal B-L extension of the Standard model: Z' and neutrinos
• Learning New Physics from a Machine
• Extending the Bump Hunt with Machine Learning
• Search for new resonances in mass distributions of jet pairs using 139 fb${}^{-1}$ of $pp$ collisions at $\sqrt{s}=13$ TeV with the ATLAS detector
• On measuring the masses of pair-produced semi-invisibly decaying particles at hadron colliders
• QCD or What?
• Novelty Detection Meets Collider Physics
• Adversarially-trained autoencoders for robust unsupervised new physics searches
• Search for high mass dijet resonances with a new background prediction method in proton-proton collisions at $\sqrt{s}=$ 13 TeV
• Dijet resonance search with weak supervision using $\sqrt{s}=13$ TeV $pp$ collisions in the ATLAS detector
• Guiding New Physics Searches with Unsupervised Learning
• Variational Autoencoders for New Physics Mining at the Large Hadron Collider
• Dijet Searches for Supersymmetry at the LHC
• Learning to Classify from Impure Samples with High-Dimensional Data
• Long-lived Heavy Neutrinos from Higgs Decays
• Weakly Supervised Classification in High Energy Physics
• On hypothesis testing, trials factor, hypertests and the BumpHunter
• Global Search for New Physics with 2.0/fb at CDF
• Nonparametric Density Estimation for High-Dimensional Data - Algorithms and Applications
• Better Latent Spaces for Better Autoencoders
• Deep Sets
• Beyond the Minimal Supersymmetric Standard Model: from theory to phenomenology
• Recursive Jigsaw Reconstruction: HEP event analysis in the presence of kinematic and combinatoric ambiguities
• Model-Independent and Quasi-Model-Independent Search for New Physics at CDF
• Search for New Physics in e mu X Data at D0 Using Sleuth: A Quasi-Model-Independent Search Strategy for New Physics
• (Machine) Learning to Do More with Less
• LHC Constraints on a $B-L$ Gauge Model using Contur
• Rare and Different: Anomaly Scores from a combination of likelihood and out-of-distribution models to detect new physics at the LHC
• A general search for new phenomena in ep scattering at HERA
• Deep Set Prediction Networks
• A strategy for a general search for new phenomena using data-derived signal regions and its application within the ATLAS experiment
• A Quasi-Model-Independent Search for New Physics at Large Transverse Momentum
• Phase Space Sampling and Inference from Weighted Events with Autoregressive Flows
• A General Search for New Phenomena at HERA
• Deep Set Auto Encoders for Anomaly Detection in Particle Physics
• Measurement of the $\mathrm{t}\bar{\mathrm{t}}\mathrm{b}\bar{\mathrm{b}}$ production cross section in the all-jet final state in pp collisions at $\sqrt{s}=$ 13 TeV
• A Quasi-Model-Independent Search for New High p_T Physics at DZero
• Topological Obstructions to Autoencoding
• Discovering the Significance of 5 sigma
• MUSiC: a model-unspecific search for new physics in proton-proton collisions at $\sqrt{s}=$ 13 TeV
• A comparison of optimisation algorithms for high-dimensional particle and astrophysics applications
• Identification of point sources in gamma rays using U-shaped convolutional neural networks and a data challenge
• Model independent search for new phenomena in pp(bar) collisions at sqrt(s) =1.96 TeV
• Convolutional Normalizing Flows

Cited by in corpus (61)

• Anomaly detection in high-energy physics using a quantum autoencoder
• Classifying Anomalies THrough Outer Density Estimation (CATHODE)
• Autoencoders on FPGAs for real-time, unsupervised new physics detection at 40 MHz at the Large Hadron Collider
• Machine Learning for Anomaly Detection in Particle Physics
• Rare and Different: Anomaly Scores from a combination of likelihood and out-of-distribution models to detect new physics at the LHC
• Challenges for Unsupervised Anomaly Detection in Particle Physics
• Resonant anomaly detection without background sculpting
• Autoencoders for Semivisible Jet Detection
• PC-JeDi: Diffusion for Particle Cloud Generation in High Energy Physics
• Online-compatible Unsupervised Non-resonant Anomaly Detection
• Deep Set Auto Encoders for Anomaly Detection in Particle Physics
• FETA: Flow-Enhanced Transportation for Anomaly Detection
• Quantum Anomaly Detection for Collider Physics
• Improving Variational Autoencoders for New Physics Detection at the LHC with Normalizing Flows
• Quantum anomaly detection in the latent space of proton collision events at the LHC
• Unravelling physics beyond the standard model with classical and quantum anomaly detection
• Self-supervised Anomaly Detection for New Physics
• Learning new physics efficiently with nonparametric methods
• Search for new phenomena in two-body invariant mass distributions using unsupervised machine learning for anomaly detection at $\sqrt{s}=13$ TeV with the ATLAS detector
• Anomaly detection search for new resonances decaying into a Higgs boson and a generic new particle $X$ in hadronic final states using $\sqrt{s}=13$ TeV $pp$ collisions with the ATLAS detector
• The Interplay of Machine Learning--based Resonant Anomaly Detection Methods
• Deep Generative Models for Detector Signature Simulation: A Taxonomic Review
• What's Anomalous in LHC Jets?
• Shared Data and Algorithms for Deep Learning in Fundamental Physics
• Anomaly Detection under Coordinate Transformations
• Using a nested anomaly detection machine learning algorithm to study the neutral triple gauge couplings at an \texorpdfstring{$e^{+}{e}^{-}$}{e+e-} collider
• High-dimensional and Permutation Invariant Anomaly Detection
• Nanosecond anomaly detection with decision trees and real-time application to exotic Higgs decays
• Boosting mono-jet searches with model-agnostic machine learning
• A Method to Simultaneously Facilitate All Jet Physics Tasks
• Quantum Generative Adversarial Networks For Anomaly Detection In High Energy Physics
• Data-Directed Search for New Physics based on Symmetries of the SM
• Anomalies, Representations, and Self-Supervision
• Combining Resonant and Tail-based Anomaly Detection
• Event-based anomaly detection for new physics searches at the LHC using machine learning
• Benchmarking energy consumption and latency for neuromorphic computing in condensed matter and particle physics
• Resonant Anomaly Detection with Multiple Reference Datasets
• Enhancing the hunt for new phenomena in dijet final-states using anomaly detection filters at the High-Luminosity Large Hadron Collider
• Mixture-of-theories Training: Can We Find New Physics and Anomalies Better by Mixing Physical Theories?
• Large Physics Models: Towards a collaborative approach with Large Language Models and Foundation Models
• Search for new physics in final states with semi-visible jets or anomalous signatures using the ATLAS detector
• Deep learning techniques for Imaging Air Cherenkov Telescopes
• Anomaly detection with flow-based fast calorimeter simulators
• Semi-visible jets, energy-based models, and self-supervision
• High-dimensional Anomaly Detection with Radiative Return in $e^{+}{e}^{-}$ Collisions
• Non-resonant Anomaly Detection with Background Extrapolation
• Simulation-based Anomaly Detection for Multileptons at the LHC
• Attention to the strengths of physical interactions: Transformer and graph-based event classification for particle physics experiments
• Isolating Unisolated Upsilons with Anomaly Detection in CMS Open Data
• Foundations of automatic feature extraction at LHC--point clouds and graphs
• Weakly supervised anomaly detection for resonant new physics in the dijet final state using proton-proton collisions at $\sqrt{s}=13$ TeV with the ATLAS detector
• Semi-supervised permutation invariant particle-level anomaly detection
• Decorrelation using Optimal Transport
• OmniLearned: A Foundation Model Framework for All Tasks Involving Jet Physics
• Event Generation and Density Estimation with Surjective Normalizing Flows
• Universal New Physics Latent Space
• Graph theory inspired anomaly detection at the LHC
• Weakly supervised anomaly detection with event-level variables
• Creating Simple, Interpretable Anomaly Detectors for New Physics in Jet Substructure
• It's not a FAD: first results in using Flows for unsupervised Anomaly Detection at 40 MHz at the Large Hadron Collider
• How to use Machine Learning to improve the discrimination between signal and background at particle colliders