TrackFormers: In Search of Transformer-Based Particle Tracking for the High-Luminosity LHC Era

Abstract

High-Energy Physics experiments are facing a multi-fold data increase with every new iteration. This is certainly the case for the upcoming High-Luminosity LHC upgrade. Such increased data processing requirements forces revisions to almost every step of the data processing pipeline. One such step in need of an overhaul is the task of particle track reconstruction, a.k.a., tracking. A Machine Learning-assisted solution is expected to provide significant improvements, since the most time-consuming step in tracking is the assignment of hits to particles or track candidates. This is the topic of this paper. We take inspiration from large language models. As such, we consider two approaches: the prediction of the next word in a sentence (next hit point in a track), as well as the one-shot prediction of all hits within an event. In an extensive design effort, we have experimented with three models based on the Transformer architecture and one model based on the U-Net architecture, performing track association predictions for collision event hit points. In our evaluation, we consider a spectrum of simple to complex representations of the problem, eliminating designs with lower metrics early on. We report extensive results, covering both prediction accuracy (score) and computational performance. We have made use of the REDVID simulation framework, as well as reductions applied to the TrackML data set, to compose five data sets from simple to complex, for our experiments. The results highlight distinct advantages among different designs in terms of prediction accuracy and computational performance, demonstrating the efficiency of our methodology. Most importantly, the results show the viability of a one-shot encoder-classifier based Transformer solution as a practical approach for the task of tracking.

References in corpus (25)

• Longformer: The Long-Document Transformer
• Submanifold Sparse Convolutional Networks
• Graph Neural Networks in Particle Physics
• FlashAttention-2: Faster Attention with Better Parallelism and Work Partitioning
• A Common Tracking Software Project
• Graph Neural Networks for Particle Reconstruction in High Energy Physics detectors
• Point Cloud Transformers applied to Collider Physics
• PC-JeDi: Diffusion for Particle Cloud Generation in High Energy Physics
• Graph Neural Networks for Particle Tracking and Reconstruction
• Semantic Segmentation with a Sparse Convolutional Neural Network for Event Reconstruction in MicroBooNE
• Object condensation: one-stage grid-free multi-object reconstruction in physics detectors, graph and image data
• Graph Neural Networks for Charged Particle Tracking on FPGAs
• Accelerated Charged Particle Tracking with Graph Neural Networks on FPGAs
• Jet Diffusion versus JetGPT -- Modern Networks for the LHC
• Towards a realistic track reconstruction algorithm based on graph neural networks for the HL-LHC
• Speeding up Particle Track Reconstruction using a Parallel Kalman Filter Algorithm
• Deep learning for track recognition in pixel and strip-based particle detectors
• Progress in developing a hybrid deep learning algorithm for identifying and locating primary vertices
• HyperTrack: Neural Combinatorics for High Energy Physics
• Comparing and improving hybrid deep learning algorithms for identifying and locating primary vertices
• High Pileup Particle Tracking with Object Condensation
• Reduced Simulations for High-Energy Physics, a Middle Ground for Data-Driven Physics Research
• A Language Model for Particle Tracking
• Universal Anomaly Detection at the LHC: Transforming Optimal Classifiers and the DDD Method
• Ranking-based neural network for ambiguity resolution in ACTS