OmniLearned: A Foundation Model Framework for All Tasks Involving Jet Physics

Abstract

Foundation models use large datasets to build an effective representation of data that can be deployed on diverse downstream tasks. Previous research developed the OmniLearn foundation model for jet physics, using unique properties of particle physics, and showed that it could significantly advance discovery potential across collider experiments. This paper introduces a major upgrade, resulting in the OmniLearned framework. This framework has three new elements: (1) updates to the model architecture and training, (2) using over one billion jets used for training, and (3) providing well-documented software for accessing all datasets and models. We demonstrate OmniLearned with three representative tasks: top-quark jet tagging with the community Delphes-based benchmark dataset, b-tagging with ATLAS full simulation, and anomaly detection with CMS experimental data. In each case, OmniLearned is the state of the art, further expanding the discovery potential of past, current, and future collider experiments.

12 pages, 5 figures

References in corpus (54)

• PYTHIA 6.4 Physics and Manual
• PYTHIA 6.2 Physics and Manual
• The automated computation of tree-level and next-to-leading order differential cross sections, and their matching to parton shower simulations
• The anti-k_t jet clustering algorithm
• An Introduction to PYTHIA 8.2
• FastJet user manual
• DELPHES 3, A modular framework for fast simulation of a generic collider experiment
• A general framework for implementing NLO calculations in shower Monte Carlo programs: the POWHEG BOX
• A New Method for Combining NLO QCD with Shower Monte Carlo Algorithms
• Matching NLO QCD computations with Parton Shower simulations: the POWHEG method
• Parton distributions with LHC data
• The ATLAS Simulation Infrastructure
• Dispelling the N^3 myth for the Kt jet-finder
• A Positive-Weight Next-to-Leading-Order Monte Carlo for Heavy Flavour Hadroproduction
• Soft Drop
• ParticleNet: Jet Tagging via Particle Clouds
• Jet Substructure at the Large Hadron Collider: A Review of Recent Advances in Theory and Machine Learning
• Anomaly Detection with Density Estimation
• Jet Substructure at the Large Hadron Collider: Experimental Review
• Classification without labels: Learning from mixed samples in high energy physics
• Search for new particles decaying into dijets in proton-antiproton collisions at sqrt(s) = 1.96 TeV
• Anomaly Detection for Resonant New Physics with Machine Learning
• Energy Flow Networks: Deep Sets for Particle Jets
• The Machine Learning Landscape of Top Taggers
• Extending the Bump Hunt with Machine Learning
• Search for New Phenomena in Dijet Mass and Angular Distributions from $pp$ Collisions at $\sqrt{s}$ = 13 TeV with the ATLAS Detector
• Dijet resonance search with weak supervision using $\sqrt{s}=13$ TeV $pp$ collisions in the ATLAS detector
• Search for narrow resonances in dijet final states at sqrt(s) = 8 TeV with the novel CMS technique of data scouting
• JEDI-net: a jet identification algorithm based on interaction networks
• The Dark Machines Anomaly Score Challenge: Benchmark Data and Model Independent Event Classification for the Large Hadron Collider
• An Efficient Lorentz Equivariant Graph Neural Network for Jet Tagging
• SoftKiller, a particle-level pileup removal method
• Classifying Anomalies THrough Outer Density Estimation (CATHODE)
• ABCNet: An attention-based method for particle tagging
• Particle-level pileup subtraction for jets and jet shapes
• Fast Point Cloud Generation with Diffusion Models in High Energy Physics
• Point Cloud Transformers applied to Collider Physics
• Optimisation of large-radius jet reconstruction for the ATLAS detector in 13 TeV proton-proton collisions
• Via Machinae: Searching for Stellar Streams using Unsupervised Machine Learning
• Pileup and Underlying Event Mitigation with Iterative Constituent Subtraction
• OmniJet-$\alpha$: The first cross-task foundation model for particle physics
• Masked Particle Modeling on Sets: Towards Self-Supervised High Energy Physics Foundation Models
• Model-agnostic search for dijet resonances with anomalous jet substructure in proton-proton collisions at $\sqrt{s}$ = 13 TeV
• Solving Key Challenges in Collider Physics with Foundation Models
• A Method to Simultaneously Facilitate All Jet Physics Tasks
• Re-Simulation-based Self-Supervised Learning for Pre-Training Foundation Models
• A Lorentz-Equivariant Transformer for All of the LHC
• Jet Tagging with More-Interaction Particle Transformer
• Preservation through modernisation: The software of the H1 experiment at HERA
• Accuracy versus precision in boosted top tagging with the ATLAS detector
• Weakly-Supervised Anomaly Detection in the Milky Way
• Aspen Open Jets: Unlocking LHC Data for Foundation Models in Particle Physics
• Isolating Unisolated Upsilons with Anomaly Detection in CMS Open Data
• Weakly supervised anomaly detection for resonant new physics in the dijet final state using proton-proton collisions at $\sqrt{s}=13$ TeV with the ATLAS detector