Distilling particle knowledge for fast reconstruction at high-energy physics experiments

Abstract

Knowledge distillation is a form of model compression that allows artificial neural networks of different sizes to learn from one another. Its main application is the compactification of large deep neural networks to free up computational resources, in particular on edge devices. In this article, we consider proton-proton collisions at the High-Luminosity LHC (HL-LHC) and demonstrate a successful knowledge transfer from an event-level graph neural network (GNN) to a particle-level small deep neural network (DNN). Our algorithm, DistillNet, is a DNN that is trained to learn about the provenance of particles, as provided by the soft labels that are the GNN outputs, to predict whether or not a particle originates from the primary interaction vertex. The results indicate that for this problem, which is one of the main challenges at the HL-LHC, there is minimal loss during the transfer of knowledge to the small student network, while improving significantly the computational resource needs compared to the teacher. This is demonstrated for the distilled student network on a CPU, as well as for a quantized and pruned student network deployed on a field-programmable gate array. Our study proves that knowledge transfer between networks of different complexity can be used for fast artificial intelligence (AI) in high-energy physics that improves the expressiveness of observables over non-AI-based reconstruction algorithms. Such an approach can become essential at the HL-LHC experiments, e.g., to comply with the resource budget of their trigger stages.

12 pages, 5 figures, updated with published version

References in corpus (17)

• Distilling the Knowledge in a Neural Network
• The anti-k_t jet clustering algorithm
• An Introduction to PYTHIA 8.2
• DELPHES 3, A modular framework for fast simulation of a generic collider experiment
• Fast inference of deep neural networks in FPGAs for particle physics
• Pileup Per Particle Identification
• Interleaved Parton Showers and Tuning Prospects
• Automatic heterogeneous quantization of deep neural networks for low-latency inference on the edge for particle detectors
• An Efficient Lorentz Equivariant Graph Neural Network for Jet Tagging
• ABCNet: An attention-based method for particle tagging
• Graph Neural Networks for Particle Reconstruction in High Energy Physics detectors
• MLPF: Efficient machine-learned particle-flow reconstruction using graph neural networks
• Point Cloud Transformers applied to Collider Physics
• Pile-Up Mitigation using Attention
• Semi-supervised Graph Neural Networks for Pileup Noise Removal
• Optimal transport for a novel event description at hadron colliders
• Efficient and Robust Jet Tagging at the LHC with Knowledge Distillation