Anomalies, Representations, and Self-Supervision

Abstract

We develop a self-supervised method for density-based anomaly detection using contrastive learning, and test it using event-level anomaly data from CMS ADC2021. The AnomalyCLR technique is data-driven and uses augmentations of the background data to mimic non-Standard-Model events in a model-agnostic way. It uses a permutation-invariant Transformer Encoder architecture to map the objects measured in a collider event to the representation space, where the data augmentations define a representation space which is sensitive to potential anomalous features. An AutoEncoder trained on background representations then computes anomaly scores for a variety of signals in the representation space. With AnomalyCLR we find significant improvements on performance metrics for all signals when compared to the raw data baseline.

19 pages, 3 figures, journal version

References in corpus (63)

• A Simple Framework for Contrastive Learning of Visual Representations
• The anti-k_t jet clustering algorithm
• An Introduction to PYTHIA 8.2
• FastJet user manual
• DELPHES 3, A modular framework for fast simulation of a generic collider experiment
• ParticleNet: Jet Tagging via Particle Clouds
• Anomaly Detection with Density Estimation
• Classification without labels: Learning from mixed samples in high energy physics
• CSI: Novelty Detection via Contrastive Learning on Distributionally Shifted Instances
• Searching for New Physics with Deep Autoencoders
• Anomaly Detection for Resonant New Physics with Machine Learning
• The Machine Learning Landscape of Top Taggers
• Extending the Bump Hunt with Machine Learning
• QCD or What?
• Simulation Assisted Likelihood-free Anomaly Detection
• Adversarially-trained autoencoders for robust unsupervised new physics searches
• Dijet resonance search with weak supervision using $\sqrt{s}=13$ TeV $pp$ collisions in the ATLAS detector
• Variational Autoencoders for New Physics Mining at the Large Hadron Collider
• The Dark Machines Anomaly Score Challenge: Benchmark Data and Model Independent Event Classification for the Large Hadron Collider
• Anomaly detection in high-energy physics using a quantum autoencoder
• Uncovering latent jet substructure
• Classifying Anomalies THrough Outer Density Estimation (CATHODE)
• Tag N' Train: A Technique to Train Improved Classifiers on Unlabeled Data
• Anomaly detection with Convolutional Graph Neural Networks
• Self-Supervised Anomaly Detection in Computer Vision and Beyond: A Survey and Outlook
• Better Latent Spaces for Better Autoencoders
• Variational Autoencoders for Anomalous Jet Tagging
• A robust anomaly finder based on autoencoders
• Bump Hunting in Latent Space
• Rare and Different: Anomaly Scores from a combination of likelihood and out-of-distribution models to detect new physics at the LHC
• Learning the latent structure of collider events
• Mining for Strong Gravitational Lenses with Self-supervised Learning
• Challenges for Unsupervised Anomaly Detection in Particle Physics
• Quasi Anomalous Knowledge: Searching for new physics with embedded knowledge
• Resonant anomaly detection without background sculpting
• Autoencoders for Semivisible Jet Detection
• CURTAINs for your Sliding Window: Constructing Unobserved Regions by Transforming Adjacent Intervals
• Online-compatible Unsupervised Non-resonant Anomaly Detection
• Particle Transformer for Jet Tagging
• Symmetries, Safety, and Self-Supervision
• Deep Set Auto Encoders for Anomaly Detection in Particle Physics
• FETA: Flow-Enhanced Transportation for Anomaly Detection
• Improving Variational Autoencoders for New Physics Detection at the LHC with Normalizing Flows
• Unsupervised Hadronic SUEP at the LHC
• Anomalous Jet Identification via Sequence Modeling
• Modern Machine Learning for LHC Physicists
• Self-supervised Anomaly Detection for New Physics
• What's Anomalous in LHC Jets?
• Lorentz group equivariant autoencoders
• A method to challenge symmetries in data with self-supervised learning
• Anomaly Detection under Coordinate Transformations
• IRC-safe Graph Autoencoder for unsupervised anomaly detection
• Boosting mono-jet searches with model-agnostic machine learning
• Mean-Shifted Contrastive Loss for Anomaly Detection
• Null Hypothesis Test for Anomaly Detection
• Resonant Anomaly Detection with Multiple Reference Datasets
• PELICAN: Permutation Equivariant and Lorentz Invariant or Covariant Aggregator Network for Particle Physics
• Estimating Galactic Distances From Images Using Self-supervised Representation Learning
• Invariant Representation Driven Neural Classifier for Anti-QCD Jet Tagging
• A Normalized Autoencoder for LHC Triggers
• Exploring Optimal Transport for Event-Level Anomaly Detection at the Large Hadron Collider
• Self-supervised component separation for the extragalactic submillimeter sky
• Anomaly Detection With Conditional Variational Autoencoders

Cited by in corpus (9)

• Masked Particle Modeling on Sets: Towards Self-Supervised High Energy Physics Foundation Models
• Solving Key Challenges in Collider Physics with Foundation Models
• A Method to Simultaneously Facilitate All Jet Physics Tasks
• Semi-visible jets, energy-based models, and self-supervision
• Unsupervised and lightly supervised learning in particle physics
• Semi-supervised permutation invariant particle-level anomaly detection
• Fine-tuning machine-learned particle-flow reconstruction for new detector geometries in future colliders
• Anomaly preserving contrastive neural embeddings for end-to-end model-independent searches at the LHC
• Large Language Models -- the Future of Fundamental Physics?