FINN: A Framework for Fast, Scalable Binarized Neural Network Inference

Abstract

Research has shown that convolutional neural networks contain significant redundancy, and high classification accuracy can be obtained even when weights and activations are reduced from floating point to binary values. In this paper, we present FINN, a framework for building fast and flexible FPGA accelerators using a flexible heterogeneous streaming architecture. By utilizing a novel set of optimizations that enable efficient mapping of binarized neural networks to hardware, we implement fully connected, convolutional and pooling layers, with per-layer compute resources being tailored to user-provided throughput requirements. On a ZC706 embedded FPGA platform drawing less than 25 W total system power, we demonstrate up to 12.3 million image classifications per second with 0.31 μs latency on the MNIST dataset with 95.8% accuracy, and 21906 image classifications per second with 283 μs latency on the CIFAR-10 and SVHN datasets with respectively 80.1% and 94.9% accuracy. To the best of our knowledge, ours are the fastest classification rates reported to date on these benchmarks.

To appear in the 25th International Symposium on Field-Programmable Gate Arrays, February 2017

References in corpus (7)

• Very Deep Convolutional Networks for Large-Scale Image Recognition
• Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift
• Deep Learning in Neural Networks: An Overview
• Binarized Neural Networks: Training Deep Neural Networks with Weights and Activations Constrained to +1 or -1
• DoReFa-Net: Training Low Bitwidth Convolutional Neural Networks with Low Bitwidth Gradients
• PlaNet - Photo Geolocation with Convolutional Neural Networks
• Resiliency of Deep Neural Networks under Quantization

Cited by in corpus (84)

• Binary Neural Networks: A Survey
• FPGA-based Accelerators of Deep Learning Networks for Learning and Classification: A Review
• Automatic heterogeneous quantization of deep neural networks for low-latency inference on the edge for particle detectors
• CirCNN: Accelerating and Compressing Deep Neural Networks Using Block-CirculantWeight Matrices
• Enable Deep Learning on Mobile Devices: Methods, Systems, and Applications
• XNOR Neural Engine: a Hardware Accelerator IP for 21.6 fJ/op Binary Neural Network Inference
• Fast convolutional neural networks on FPGAs with hls4ml
• Compressing deep neural networks on FPGAs to binary and ternary precision with HLS4ML
• Deep Learning for Automated Experimentation in Scanning Transmission Electron Microscopy
• Machine Learning for the Control and Monitoring of Electric Machine Drives: Advances and Trends
• A Survey on Deep Learning Hardware Accelerators for Heterogeneous HPC Platforms
• Synetgy: Algorithm-hardware Co-design for ConvNet Accelerators on Embedded FPGAs
• Layer-specific Optimization for Mixed Data Flow with Mixed Precision in FPGA Design for CNN-based Object Detectors
• PointAcc: Efficient Point Cloud Accelerator
• Deep Neural Network Approximation for Custom Hardware: Where We've Been, Where We're Going
• Applications and Techniques for Fast Machine Learning in Science
• Autoencoders on FPGAs for real-time, unsupervised new physics detection at 40 MHz at the Large Hadron Collider
• Understanding the Potential of FPGA-Based Spatial Acceleration for Large Language Model Inference
• NEURAghe: Exploiting CPU-FPGA Synergies for Efficient and Flexible CNN Inference Acceleration on Zynq SoCs
• AddNet: Deep Neural Networks Using FPGA-Optimized Multipliers
• Tactics to Directly Map CNN graphs on Embedded FPGAs
• Charged particle tracking via edge-classifying interaction networks
• FPGA-Based CNN Inference Accelerator Synthesized from Multi-Threaded C Software
• Synergy: A HW/SW Framework for High Throughput CNNs on Embedded Heterogeneous SoC
• Inference of Quantized Neural Networks on Heterogeneous All-Programmable Devices
• CFU Playground: Full-Stack Open-Source Framework for Tiny Machine Learning (tinyML) Acceleration on FPGAs
• E3NE: An End-to-End Framework for Accelerating Spiking Neural Networks with Emerging Neural Encoding on FPGAs
• A Stochastic-Computing based Deep Learning Framework using Adiabatic Quantum-Flux-Parametron SuperconductingTechnology
• Efficient Error-Tolerant Quantized Neural Network Accelerators
• PolyLUT: Learning Piecewise Polynomials for Ultra-Low Latency FPGA LUT-based Inference
• Streaming Message Interface: High-Performance Distributed Memory Programming on Reconfigurable Hardware
• Streaming Architecture for Large-Scale Quantized Neural Networks on an FPGA-Based Dataflow Platform
• Binary Neural Networks for Memory-Efficient and Effective Visual Place Recognition in Changing Environments
• Throughput Optimizations for FPGA-based Deep Neural Network Inference
• A Lightweight FPGA-based IDS-ECU Architecture for Automotive CAN
• Generic and Universal Parallel Matrix Summation with a Flexible Compression Goal for Xilinx FPGAs
• A Lightweight Multi-Attack CAN Intrusion Detection System on Hybrid FPGAs
• DeepFire2: A Convolutional Spiking Neural Network Accelerator on FPGAs
• Accelerating Generalized Linear Models with MLWeaving: A One-Size-Fits-All System for Any-precision Learning (Technical Report)
• Optimizing Bit-Serial Matrix Multiplication for Reconfigurable Computing
• TreeLUT: An Efficient Alternative to Deep Neural Networks for Inference Acceleration Using Gradient Boosted Decision Trees
• Optimized Partitioning and Priority Assignment of Real-Time Applications on Heterogeneous Platforms with Hardware Acceleration
• NeuraLUT: Hiding Neural Network Density in Boolean Synthesizable Functions
• HCM: Hardware-Aware Complexity Metric for Neural Network Architectures
• HIDA: A Hierarchical Dataflow Compiler for High-Level Synthesis
• LPYOLO: Low Precision YOLO for Face Detection on FPGA
• Field-Programmable Gate Array Architecture for Deep Learning: Survey & Future Directions
• An Encoding Framework for Binarized Images using HyperDimensional Computing
• Stream-HLS: Towards Automatic Dataflow Acceleration
• A Configurable BNN ASIC using a Network of Programmable Threshold Logic Standard Cells
• HALF: Holistic Auto Machine Learning for FPGAs
• HARFLOW3D: A Latency-Oriented 3D-CNN Accelerator Toolflow for HAR on FPGA Devices
• Logic Shrinkage: Learned FPGA Netlist Sparsity for Efficient Neural Network Inference
• Exploring Highly Quantised Neural Networks for Intrusion Detection in Automotive CAN
• Real-time Graph Building on FPGAs for Machine Learning Trigger Applications in Particle Physics
• Diagnostic data integration using deep neural networks for real-time plasma analysis
• MajorityNets: BNNs Utilising Approximate Popcount for Improved Efficiency
• FLOWER: A comprehensive dataflow compiler for high-level synthesis
• Dynamically Reconfigurable Variable-precision Sparse-Dense Matrix Acceleration in Tensorflow Lite
• Evolutionary Bin Packing for Memory-Efficient Dataflow Inference Acceleration on FPGA
• MetaML: Automating Customizable Cross-Stage Design-Flow for Deep Learning Acceleration
• Accelerating Deep Neuroevolution on Distributed FPGAs for Reinforcement Learning Problems
• Rethinking Arithmetic for Deep Neural Networks
• B-DCGAN:Evaluation of Binarized DCGAN for FPGA
• LL-ViT: Edge Deployable Vision Transformers with Look Up Table Neurons
• FPGA Processor In Memory Architectures (PIMs): Overlay or Overhaul ?
• Towards real-time and energy efficient Siamese tracking -- a hardware-software approach
• Quantised Neural Network Accelerators for Low-Power IDS in Automotive Networks
• SILVIA: Automated Superword-Level Parallelism Exploitation via HLS-Specific LLVM Passes for Compute-Intensive FPGA Accelerators
• PowerYOLO: Mixed Precision Model for Hardware Efficient Object Detection with Event Data
• Scalar Arithmetic Multiple Data: Customizable Precision for Deep Neural Networks
• Side-Channel Extraction of Dataflow AI Accelerator Hardware Parameters
• Exploration of Hardware Acceleration Methods for an XNOR Traffic Signs Classifier
• CGRA4ML: A Hardware/Software Framework to Implement Neural Networks for Scientific Edge Computing
• A Hybrid Approach combining ANN-based and Conventional Demapping in Communication for Efficient FPGA-Implementation
• When Side-Channel Attacks Break the Black-Box Property of Embedded Artificial Intelligence
• Evaluating Four FPGA-accelerated Space Use Cases based on Neural Network Algorithms for On-board Inference
• A DSP shared is a DSP earned: HLS Task-Level Multi-Pumping for High-Performance Low-Resource Designs
• JEDI-linear: Fast and Efficient Graph Neural Networks for Jet Tagging on FPGAs
• Compact and Efficient Neural Networks for Image Recognition Based on Learned 2D Separable Transform
• Efficient Logic Gate Networks for Video Copy Detection
• Bitwise Systolic Array Architecture for Runtime-Reconfigurable Multi-precision Quantized Multiplication on Hardware Accelerators
• Bit-Width-Aware Design Environment for Few-Shot Learning on Edge AI Hardware
• RISC-V and machine learning: a survey