Learning the language of QCD jets with transformers

Abstract

Transformers have become the primary architecture for natural language processing. In this study, we explore their use for auto-regressive density estimation in high-energy jet physics, which involves working with a high-dimensional space. We draw an analogy between sentences and words in natural language and jets and their constituents in high-energy physics. Specifically, we investigate density estimation for light QCD jets and hadronically decaying boosted top jets. Since transformers allow easy sampling from learned densities, we exploit their generative capability to assess the quality of the density estimate. Our results indicate that the generated data samples closely resemble the original data, as evidenced by the excellent agreement of distributions such as particle multiplicity or jet mass. Furthermore, the generated samples are difficult to distinguish from the original data, even by a powerful supervised classifier. Given their exceptional data processing capabilities, transformers could potentially be trained directly on the massive LHC data sets to learn the probability densities in high-energy jet physics.

arXiv commentFew references added; Version accepted for publication by JHEP

References in corpus (10)

• ParticleNet: Jet Tagging via Particle Clouds
• Anomaly Detection with Density Estimation
• The Machine Learning Landscape of Top Taggers
• QCD-Aware Recursive Neural Networks for Jet Physics
• JUNIPR: a Framework for Unsupervised Machine Learning in Particle Physics
• Machine Learning and LHC Event Generation
• Evaluating generative models in high energy physics
• Point Cloud Transformers applied to Collider Physics
• Symmetries, Safety, and Self-Supervision
• Binary JUNIPR: an interpretable probabilistic model for discrimination

Cited by in corpus (15)

• OmniJet-$\alpha$: The first cross-task foundation model for particle physics
• Deep Generative Models for Detector Signature Simulation: A Taxonomic Review
• Masked Particle Modeling on Sets: Towards Self-Supervised High Energy Physics Foundation Models
• High-dimensional and Permutation Invariant Anomaly Detection
• Precision-Machine Learning for the Matrix Element Method
• A Method to Simultaneously Facilitate All Jet Physics Tasks
• Equivariant, Safe and Sensitive -- Graph Networks for New Physics
• Interpretable deep learning models for the inference and classification of LHC data
• Learning the Simplicity of Scattering Amplitudes
• Unifying Simulation and Inference with Normalizing Flows
• Foundations of automatic feature extraction at LHC--point clouds and graphs
• Extrapolating Jet Radiation with Autoregressive Transformers
• Tagging fully hadronic exotic decays of the vectorlike $\mathbf{B}$ quark using a graph neural network
• What exactly did the Transformer learn from our physics data?
• Large Language Models -- the Future of Fundamental Physics?