FPGA-accelerated machine learning inference as a service for particle physics computing

Abstract

New heterogeneous computing paradigms on dedicated hardware with increased parallelization, such as Field Programmable Gate Arrays (FPGAs), offer exciting solutions with large potential gains. The growing applications of machine learning algorithms in particle physics for simulation, reconstruction, and analysis are naturally deployed on such platforms. We demonstrate that the acceleration of machine learning inference as a web service represents a heterogeneous computing solution for particle physics experiments that potentially requires minimal modification to the current computing model. As examples, we retrain the ResNet-50 convolutional neural network to demonstrate state-of-the-art performance for top quark jet tagging at the LHC and apply a ResNet-50 model with transfer learning for neutrino event classification. Using Project Brainwave by Microsoft to accelerate the ResNet-50 image classification model, we achieve average inference times of 60 (10) milliseconds with our experimental physics software framework using Brainwave as a cloud (edge or on-premises) service, representing an improvement by a factor of approximately 30 (175) in model inference latency over traditional CPU inference in current experimental hardware. A single FPGA service accessed by many CPUs achieves a throughput of 600--700 inferences per second using an image batch of one, comparable to large batch-size GPU throughput and significantly better than small batch-size GPU throughput. Deployed as an edge or cloud service for the particle physics computing model, coprocessor accelerators can have a higher duty cycle and are potentially much more cost-effective.

16 pages, 14 figures, 2 tables

References in corpus (21)

• Adam: A Method for Stochastic Optimization
• Very Deep Convolutional Networks for Large-Scale Image Recognition
• Observation of a new particle in the search for the Standard Model Higgs boson with the ATLAS detector at the LHC
• Observation of a new boson at a mass of 125 GeV with the CMS experiment at the LHC
• The anti-k_t jet clustering algorithm
• An Introduction to PYTHIA 8.2
• FastJet user manual
• DELPHES 3, A modular framework for fast simulation of a generic collider experiment
• Dispelling the N^3 myth for the Kt jet-finder
• Tuning PYTHIA 8.1: the Monash 2013 Tune
• Fast inference of deep neural networks in FPGAs for particle physics
• ParticleNet: Jet Tagging via Particle Clouds
• CaloGAN: Simulating 3D High Energy Particle Showers in Multi-Layer Electromagnetic Calorimeters with Generative Adversarial Networks
• Reionization and the Cosmic Dawn with the Square Kilometre Array
• A Convolutional Neural Network Neutrino Event Classifier
• A Roadmap for HEP Software and Computing R&D for the 2020s
• Energy calibration and resolution of the CMS electromagnetic calorimeter in pp collisions at sqrt(s) = 7 TeV
• Deep-learned Top Tagging with a Lorentz Layer
• Constraints on Oscillation Parameters from ${\nu }_e$ Appearance and ${\nu }_{\mu }$ Disappearance in NOvA
• Convolutional Neural Networks Applied to Neutrino Events in a Liquid Argon Time Projection Chamber
• Long-Baseline Neutrino Facility (LBNF) and Deep Underground Neutrino Experiment (DUNE) Conceptual Design Report Volume 1: The LBNF and DUNE Projects

Cited by in corpus (15)

• Applications and Techniques for Fast Machine Learning in Science
• MLPF: Efficient machine-learned particle-flow reconstruction using graph neural networks
• Casting a graph net to catch dark showers
• Machine Learning in High Energy Physics: A review of heavy-flavor jet tagging at the LHC
• GPU-accelerated machine learning inference as a service for computing in neutrino experiments
• FPGAs-as-a-Service Toolkit (FaaST)
• GPU coprocessors as a service for deep learning inference in high energy physics
• Accelerating the search for Axion-Like Particles with machine learning
• Accelerating Deep Neural Networks for Real-time Data Selection for High-resolution Imaging Particle Detectors
• Fast Neural Network Inference on FPGAs for Triggering on Long-Lived Particles at Colliders
• Portable acceleration of CMS computing workflows with coprocessors as a service
• Development of a resource-efficient FPGA-based neural network regression model for the ATLAS muon trigger upgrades
• Summary of the trigger systems of the Large Hadron Collider experiments ALICE, ATLAS, CMS and LHCb
• The SMARTHEP European Training Network
• SuperSONIC: Cloud-Native Infrastructure for ML Inferencing