Full-stack Optimization for Accelerating CNNs with FPGA Validation

Abstract

We present a full-stack optimization framework for accelerating inference of CNNs (Convolutional Neural Networks) and validate the approach with field-programmable gate arrays (FPGA) implementations. By jointly optimizing CNN models, computing architectures, and hardware implementations, our full-stack approach achieves unprecedented performance in the trade-off space characterized by inference latency, energy efficiency, hardware utilization and inference accuracy. As a validation vehicle, we have implemented a 170MHz FPGA inference chip achieving 2.28ms latency for the ImageNet benchmark. The achieved latency is among the lowest reported in the literature while achieving comparable accuracy. However, our chip shines in that it has 9x higher energy efficiency compared to other implementations achieving comparable latency. A highlight of our full-stack approach which attributes to the achieved high energy efficiency is an efficient Selector-Accumulator (SAC) architecture for implementing the multiplier-accumulator (MAC) operation present in any digital CNN hardware. For instance, compared to a FPGA implementation for a traditional 8-bit MAC, SAC substantially reduces required hardware resources (4.85x fewer Look-up Tables) and power consumption (2.48x).

References in corpus (16)

• Very Deep Convolutional Networks for Large-Scale Image Recognition
• MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applications
• Binarized Neural Networks: Training Deep Neural Networks with Weights and Activations Constrained to +1 or -1
• DoReFa-Net: Training Low Bitwidth Convolutional Neural Networks with Low Bitwidth Gradients
• Quantizing deep convolutional networks for efficient inference: A whitepaper
• Trained Ternary Quantization
• PACT: Parameterized Clipping Activation for Quantized Neural Networks
• Incremental Network Quantization: Towards Lossless CNNs with Low-Precision Weights
• Xception: Deep Learning with Depthwise Separable Convolutions
• 8-Bit Approximations for Parallelism in Deep Learning
• Block-Sparse Recurrent Neural Networks
• CondenseNet: An Efficient DenseNet using Learned Group Convolutions
• Deep Learning with Low Precision by Half-wave Gaussian Quantization
• Shift: A Zero FLOP, Zero Parameter Alternative to Spatial Convolutions
• Packing Sparse Convolutional Neural Networks for Efficient Systolic Array Implementations: Column Combining Under Joint Optimization
• Design Flow of Accelerating Hybrid Extremely Low Bit-width Neural Network in Embedded FPGA