Fast inference of deep neural networks in FPGAs for particle physics

Abstract

Recent results at the Large Hadron Collider (LHC) have pointed to enhanced physics capabilities through the improvement of the real-time event processing techniques. Machine learning methods are ubiquitous and have proven to be very powerful in LHC physics, and particle physics as a whole. However, exploration of the use of such techniques in low-latency, low-power FPGA hardware has only just begun. FPGA-based trigger and data acquisition (DAQ) systems have extremely low, sub-microsecond latency requirements that are unique to particle physics. We present a case study for neural network inference in FPGAs focusing on a classifier for jet substructure which would enable, among many other physics scenarios, searches for new dark sector particles and novel measurements of the Higgs boson. While we focus on a specific example, the lessons are far-reaching. We develop a package based on High-Level Synthesis (HLS) called hls4ml to build machine learning models in FPGAs. The use of HLS increases accessibility across a broad user community and allows for a drastic decrease in firmware development time. We map out FPGA resource usage and latency versus neural network hyperparameters to identify the problems in particle physics that would benefit from performing neural network inference with FPGAs. For our example jet substructure model, we fit well within the available resources of modern FPGAs with a latency on the scale of 100 ns.

arXiv comment22 pages, 17 figures, 2 tables, JINST revision

References in corpus (27)

• Observation of a new particle in the search for the Standard Model Higgs boson with the ATLAS detector at the LHC
• Observation of a new boson at a mass of 125 GeV with the CMS experiment at the LHC
• The automated computation of tree-level and next-to-leading order differential cross sections, and their matching to parton shower simulations
• The anti-k_t jet clustering algorithm
• An Introduction to PYTHIA 8.2
• FastJet user manual
• Parton distributions with LHC data
• Dispelling the N^3 myth for the Kt jet-finder
• Jet substructure as a new Higgs search channel at the LHC
• Tuning PYTHIA 8.1: the Monash 2013 Tune
• Soft Drop
• Identification of heavy-flavour jets with the CMS detector in pp collisions at 13 TeV
• Energy Correlation Functions for Jet Substructure
• Deep learning in color: towards automated quark/gluon jet discrimination
• Jet-Images -- Deep Learning Edition
• Accelerating Science with Generative Adversarial Networks: An Application to 3D Particle Showers in Multi-Layer Calorimeters
• Deep-learning Top Taggers or The End of QCD?
• Observation of $\mathrm{t}\overline{\mathrm{t}}$H production
• Jet Flavor Classification in High-Energy Physics with Deep Neural Networks
• New Angles on Energy Correlation Functions
• Jet-Images: Computer Vision Inspired Techniques for Jet Tagging
• Search for narrow resonances in dijet final states at sqrt(s) = 8 TeV with the novel CMS technique of data scouting
• Search for low mass vector resonances decaying into quark-antiquark pairs in proton-proton collisions at sqrt(s) = 13 TeV
• Inclusive search for a highly boosted Higgs boson decaying to a bottom quark-antiquark pair
• Novel Jet Observables from Machine Learning
• CMS Hardware Track Trigger: New Opportunities for Long-Lived Particle Searches at the HL-LHC
• The importance of calorimetry for highly-boosted jet substructure

Cited by in corpus (96)

• Machine learning and the physical sciences
• Photonics for artificial intelligence and neuromorphic computing
• Energy Flow Networks: Deep Sets for Particle Jets
• Automatic heterogeneous quantization of deep neural networks for low-latency inference on the edge for particle detectors
• Single chip photonic deep neural network with accelerated training
• funcX: A Federated Function Serving Fabric for Science
• Prospects and applications of photonic neural networks
• Adversarially-trained autoencoders for robust unsupervised new physics searches
• Fast convolutional neural networks on FPGAs with hls4ml
• Variational Autoencoders for New Physics Mining at the Large Hadron Collider
• JEDI-net: a jet identification algorithm based on interaction networks
• Deep Learning for Automated Experimentation in Scanning Transmission Electron Microscopy
• Agile SoC Development with Open ESP
• Boosting $H\to b\bar{b}$ with Machine Learning
• Artificial Neural Networks for Photonic Applications: From Algorithms to Implementation
• Fast inference of Boosted Decision Trees in FPGAs for particle physics
• Autoencoders on FPGAs for real-time, unsupervised new physics detection at 40 MHz at the Large Hadron Collider
• Applications and Techniques for Fast Machine Learning in Science
• MLPF: Efficient machine-learned particle-flow reconstruction using graph neural networks
• Machine Learning for Anomaly Detection in Particle Physics
• FPGA-accelerated machine learning inference as a service for particle physics computing
• Reducing Computational Complexity of Neural Networks in Optical Channel Equalization: From Concepts to Implementation
• A guide for deploying Deep Learning in LHC searches: How to achieve optimality and account for uncertainty
• A reconfigurable neural network ASIC for detector front-end data compression at the HL-LHC
• Point Cloud Transformers applied to Collider Physics
• Lund jet images from generative and cycle-consistent adversarial networks
• Mapping Machine-Learned Physics into a Human-Readable Space
• Charged particle tracking via edge-classifying interaction networks
• Unsupervised clustering for collider physics
• Ps and Qs: Quantization-aware pruning for efficient low latency neural network inference
• Distance-Weighted Graph Neural Networks on FPGAs for Real-Time Particle Reconstruction in High Energy Physics
• Online-compatible Unsupervised Non-resonant Anomaly Detection
• The LHC Olympics 2020: A Community Challenge for Anomaly Detection in High Energy Physics
• Reliably-stabilizing piecewise-affine neural network controllers
• Allo: A Programming Model for Composable Accelerator Design
• Graph Neural Networks for Charged Particle Tracking on FPGAs
• PolyLUT: Learning Piecewise Polynomials for Ultra-Low Latency FPGA LUT-based Inference
• Vector Boson Scattering Processes: Status and Prospects
• ESP4ML: Platform-Based Design of Systems-on-Chip for Embedded Machine Learning
• Explainable Equivariant Neural Networks for Particle Physics: PELICAN
• Fixed-Point Convolutional Neural Network for Real-Time Video Processing in FPGA
• Etalumis: Bringing Probabilistic Programming to Scientific Simulators at Scale
• Nanosecond machine learning event classification with boosted decision trees in FPGA for high energy physics
• Accelerating Recurrent Neural Networks for Gravitational Wave Experiments
• FPGAs-as-a-Service Toolkit (FaaST)
• LL-GNN: Low Latency Graph Neural Networks on FPGAs for High Energy Physics
• NeuraLUT: Hiding Neural Network Density in Boolean Synthesizable Functions
• Implementation of high-performance, sub-microsecond deep neural networks on FPGAs for trigger applications
• A Detailed Study of Interpretability of Deep Neural Network based Top Taggers
• AI for Closed-Loop Control Systems -- New Opportunities for Modeling, Designing, and Tuning Control Systems
• Random matrix analysis of deep neural network weight matrices
• Symbolic Regression on FPGAs for Fast Machine Learning Inference
• Muon trigger with fast Neural Networks on FPGA, a demonstrator
• Improving sensitivity of the ARIANNA detector by rejecting thermal noise with deep learning
• Fast Muon Tracking with Machine Learning Implemented in FPGA
• Nanosecond machine learning regression with deep boosted decision trees in FPGA for high energy physics
• Fast convolutional neural networks for identifying long-lived particles in a high-granularity calorimeter
• Extracting low energy signals from raw LArTPC waveforms using deep learning techniques -- A proof of concept
• Ultrafast jet classification on FPGAs for the HL-LHC
• Real-time Graph Building on FPGAs for Machine Learning Trigger Applications in Particle Physics
• FAIR AI Models in High Energy Physics
• SymbolNet: Neural Symbolic Regression with Adaptive Dynamic Pruning for Compression
• Beyond 4D Tracking: Using Cluster Shapes for Track Seeding
• FPGA Resource-aware Structured Pruning for Real-Time Neural Networks
• A Machine Learning Approach to the Detection of Ghosting and Scattered Light Artifacts in Dark Energy Survey Images
• FastStamp: Accelerating Neural Steganography and Digital Watermarking of Images on FPGAs
• MetaML: Automating Customizable Cross-Stage Design-Flow for Deep Learning Acceleration
• Dynamically Reconfigurable Variable-precision Sparse-Dense Matrix Acceleration in Tensorflow Lite
• Development of a resource-efficient FPGA-based neural network regression model for the ATLAS muon trigger upgrades
• Real-time Artificial Intelligence for Accelerator Control: A Study at the Fermilab Booster
• Firmware implementation of a recurrent neural network for the computation of the energy deposited in the liquid argon calorimeter of the ATLAS experiment
• Accelerating Graph-based Tracking Tasks with Symbolic Regression
• Foundations of automatic feature extraction at LHC--point clouds and graphs
• MetaML-Pro: Cross-Stage Design Flow Automation for Efficient Deep Learning Acceleration
• Boundary between noise and information applied to filtering neural network weight matrices
• Variational Autoencoders for At-Source Data Reduction and Anomaly Detection in High Energy Particle Detectors
• BitHEP -- The Limits of Low-Precision ML in HEP
• Investigating Resource-efficient Neutron/Gamma Classification ML Models Targeting eFPGAs
• CGRA4ML: A Hardware/Software Framework to Implement Neural Networks for Scientific Edge Computing
• Nanosecond hardware regression trees in FPGA at the LHC
• Revisiting Differential Verification: Equivalence Verification with Confidence
• Review of real-time data processing for collider experiments
• Neural-network-based level-1 trigger upgrade for the SuperCDMS experiment at SNOLAB
• AI-Enhanced Self-Triggering for Extensive Air Showers: Performance and FPGA Feasibility
• Continuous-variable photonic quantum extreme learning machines for fast collider-data selection
• JEDI-linear: Fast and Efficient Graph Neural Networks for Jet Tagging on FPGAs
• Energy-Efficient Real-Time 4-Stage Sleep Classification at 10-Second Resolution
• It's not a FAD: first results in using Flows for unsupervised Anomaly Detection at 40 MHz at the Large Hadron Collider
• APEIRON: composing smart TDAQ systems for high energy physics experiments
• RISC-V and machine learning: a survey
• Distilling particle knowledge for fast reconstruction at high-energy physics experiments
• Ultra Fast Calorimeter Simulation with Generative Machine Learning on FPGAs
• Illuminating all-hadronic final states with a photon: Exotic decays of the Higgs boson to four bottom quarks in vector boson fusion plus gamma at hadron colliders
• Variational Dropout Sparsification for Particle Identification speed-up
• Real-time graph neural networks on FPGAs for the Belle II electromagnetic calorimeter
• Recent Developments in Transformer Inference Deployment on FPGA Platforms: A Survey