Fast convolutional neural networks on FPGAs with hls4ml

Abstract

We introduce an automated tool for deploying ultra low-latency, low-power deep neural networks with convolutional layers on FPGAs. By extending the hls4ml library, we demonstrate an inference latency of $5\mu$s using convolutional architectures, targeting microsecond latency applications like those at the CERN Large Hadron Collider. Considering benchmark models trained on the Street View House Numbers Dataset, we demonstrate various methods for model compression in order to fit the computational constraints of a typical FPGA device used in trigger and data acquisition systems of particle detectors. In particular, we discuss pruning and quantization-aware training, and demonstrate how resource utilization can be significantly reduced with little to no loss in model accuracy. We show that the FPGA critical resource consumption can be reduced by 97% with zero loss in model accuracy, and by 99% when tolerating a 6% accuracy degradation.

arXiv comment18 pages, 18 figures, 4 tables

References in corpus (11)

• FINN: A Framework for Fast, Scalable Binarized Neural Network Inference
• FPGA-based Accelerators of Deep Learning Networks for Learning and Classification: A Review
• Fast inference of deep neural networks in FPGAs for particle physics
• Residual Networks of Residual Networks: Multilevel Residual Networks
• Performance of the CMS Level-1 trigger in proton-proton collisions at $\sqrt{s}=$ 13 TeV
• NullHop: A Flexible Convolutional Neural Network Accelerator Based on Sparse Representations of Feature Maps
• Automatic heterogeneous quantization of deep neural networks for low-latency inference on the edge for particle detectors
• Operation of the ATLAS trigger system in Run 2
• Compressing deep neural networks on FPGAs to binary and ternary precision with HLS4ML
• Fast inference of Boosted Decision Trees in FPGAs for particle physics
• Distance-Weighted Graph Neural Networks on FPGAs for Real-Time Particle Reconstruction in High Energy Physics

Cited by in corpus (24)

• Automatic heterogeneous quantization of deep neural networks for low-latency inference on the edge for particle detectors
• Autoencoders on FPGAs for real-time, unsupervised new physics detection at 40 MHz at the Large Hadron Collider
• Applications and Techniques for Fast Machine Learning in Science
• Charged particle tracking via edge-classifying interaction networks
• Online-compatible Unsupervised Non-resonant Anomaly Detection
• Graph Neural Networks for Charged Particle Tracking on FPGAs
• Nanosecond machine learning event classification with boosted decision trees in FPGA for high energy physics
• Nanosecond machine learning regression with deep boosted decision trees in FPGA for high energy physics
• SymbolNet: Neural Symbolic Regression with Adaptive Dynamic Pruning for Compression
• FPGA Resource-aware Structured Pruning for Real-Time Neural Networks
• FastStamp: Accelerating Neural Steganography and Digital Watermarking of Images on FPGAs
• Fast Neural Network Inference on FPGAs for Triggering on Long-Lived Particles at Colliders
• Real-time Artificial Intelligence for Accelerator Control: A Study at the Fermilab Booster
• Low latency optical-based mode tracking with machine learning deployed on FPGAs on a tokamak
• Data-Model-Circuit Tri-Design for Ultra-Light Video Intelligence on Edge Devices
• Label-free timing analysis of SiPM-based modularized detectors with physics-constrained deep learning
• Accelerating Graph-based Tracking Tasks with Symbolic Regression
• da4ml: Distributed Arithmetic for Real-time Neural Networks on FPGAs
• Real-Time Cell Sorting with Scalable In Situ FPGA-Accelerated Deep Learning
• BitHEP -- The Limits of Low-Precision ML in HEP
• AI-Enhanced Self-Triggering for Extensive Air Showers: Performance and FPGA Feasibility
• Scintillation pulse characterization with spectrum-inspired temporal neural networks: case studies on particle detector signals
• Illuminating all-hadronic final states with a photon: Exotic decays of the Higgs boson to four bottom quarks in vector boson fusion plus gamma at hadron colliders
• It's not a FAD: first results in using Flows for unsupervised Anomaly Detection at 40 MHz at the Large Hadron Collider