Compressing deep neural networks on FPGAs to binary and ternary precision with HLS4ML

Abstract

We present the implementation of binary and ternary neural networks in the hls4ml library, designed to automatically convert deep neural network models to digital circuits with FPGA firmware. Starting from benchmark models trained with floating point precision, we investigate different strategies to reduce the network's resource consumption by reducing the numerical precision of the network parameters to binary or ternary. We discuss the trade-off between model accuracy and resource consumption. In addition, we show how to balance between latency and accuracy by retaining full precision on a selected subset of network components. As an example, we consider two multiclass classification tasks: handwritten digit recognition with the MNIST data set and jet identification with simulated proton-proton collisions at the CERN Large Hadron Collider. The binary and ternary implementation has similar performance to the higher precision implementation while using drastically fewer FPGA resources.

arXiv commentUpdate to MLST journal version

References in corpus (2)

• FINN: A Framework for Fast, Scalable Binarized Neural Network Inference
• JEDI-net: a jet identification algorithm based on interaction networks

Cited by in corpus (24)

• Automatic heterogeneous quantization of deep neural networks for low-latency inference on the edge for particle detectors
• Fast convolutional neural networks on FPGAs with hls4ml
• Autoencoders on FPGAs for real-time, unsupervised new physics detection at 40 MHz at the Large Hadron Collider
• Applications and Techniques for Fast Machine Learning in Science
• Charged particle tracking via edge-classifying interaction networks
• Ps and Qs: Quantization-aware pruning for efficient low latency neural network inference
• Distance-Weighted Graph Neural Networks on FPGAs for Real-Time Particle Reconstruction in High Energy Physics
• Graph Neural Networks for Charged Particle Tracking on FPGAs
• PolyLUT: Learning Piecewise Polynomials for Ultra-Low Latency FPGA LUT-based Inference
• Vector Boson Scattering Processes: Status and Prospects
• Nanosecond machine learning event classification with boosted decision trees in FPGA for high energy physics
• NeuraLUT: Hiding Neural Network Density in Boolean Synthesizable Functions
• Fast Muon Tracking with Machine Learning Implemented in FPGA
• Nanosecond machine learning regression with deep boosted decision trees in FPGA for high energy physics
• Fast Neural Network Inference on FPGAs for Triggering on Long-Lived Particles at Colliders
• Real-time Artificial Intelligence for Accelerator Control: A Study at the Fermilab Booster
• Accelerating Graph-based Tracking Tasks with Symbolic Regression
• Label-free timing analysis of SiPM-based modularized detectors with physics-constrained deep learning
• AXI4MLIR: User-Driven Automatic Host Code Generation for Custom AXI-Based Accelerators
• Review of real-time data processing for collider experiments
• BitHEP -- The Limits of Low-Precision ML in HEP
• JEDI-linear: Fast and Efficient Graph Neural Networks for Jet Tagging on FPGAs
• Illuminating all-hadronic final states with a photon: Exotic decays of the Higgs boson to four bottom quarks in vector boson fusion plus gamma at hadron colliders
• It's not a FAD: first results in using Flows for unsupervised Anomaly Detection at 40 MHz at the Large Hadron Collider