Reconstructing hadronically decaying tau leptons with a jet foundation model

Abstract

The limited availability and accuracy of simulated data has motivated the use of foundation models in high energy physics, with the idea to first train a task-agnostic model on large and potentially unlabeled datasets. This enables the subsequent fine-tuning of the learned representation for specific downstream tasks, potentially requiring much smaller dataset sizes to reach the performance of models trained from scratch. We study how OmniJet-$\alpha$, one of the proposed foundation models for particle jets, can be used on a new set of tasks, and in a new dataset, in order to reconstruct hadronically decaying $\tau$ leptons. We show that the pretraining can successfully be utilized for this multi-task problem, improving the resolution of momentum reconstruction by about 50\% when the pretrained weights are fine-tuned, compared to training the model from scratch. While much work remains ahead to develop generic foundation models for high-energy physics, this early result of generalizing an existing model to a new dataset and to previously unconsidered tasks highlights the importance of testing the approaches on a diverse set of datasets and tasks.

Submission to SciPost

References in corpus (20)

• FastJet user manual
• DELPHES 3, A modular framework for fast simulation of a generic collider experiment
• Neural Discrete Representation Learning
• On the Opportunities and Risks of Foundation Models
• Lookahead Optimizer: k steps forward, 1 step back
• A Comprehensive Survey on Transfer Learning
• Particle Transformer for Jet Tagging
• OmniJet-$\alpha$: The first cross-task foundation model for particle physics
• Masked Particle Modeling on Sets: Towards Self-Supervised High Energy Physics Foundation Models
• Solving Key Challenges in Collider Physics with Foundation Models
• Re-Simulation-based Self-Supervised Learning for Pre-Training Foundation Models
• Straightening Out the Straight-Through Estimator: Overcoming Optimization Challenges in Vector Quantized Networks
• Pre-training strategy using real particle collision data for event classification in collider physics
• Is Tokenization Needed for Masked Particle Modelling?
• Specialized Foundation Models Struggle to Beat Supervised Baselines
• The International Linear Collider Technical Design Report - Volume 4: Detectors
• Data Preservation in High Energy Physics
• Tau lepton identification and reconstruction: a new frontier for jet-tagging ML algorithms
• A unified machine learning approach for reconstructing hadronically decaying tau leptons
• The \texttau{} challenge at FCC-ee

Cited by in corpus (1)

• BitHEP -- The Limits of Low-Precision ML in HEP