NullHop: A Flexible Convolutional Neural Network Accelerator Based on Sparse Representations of Feature Maps

Abstract

Convolutional neural networks (CNNs) have become the dominant neural network architecture for solving many state-of-the-art (SOA) visual processing tasks. Even though Graphical Processing Units (GPUs) are most often used in training and deploying CNNs, their power efficiency is less than 10 GOp/s/W for single-frame runtime inference. We propose a flexible and efficient CNN accelerator architecture called NullHop that implements SOA CNNs useful for low-power and low-latency application scenarios. NullHop exploits the sparsity of neuron activations in CNNs to accelerate the computation and reduce memory requirements. The flexible architecture allows high utilization of available computing resources across kernel sizes ranging from 1x1 to 7x7. NullHop can process up to 128 input and 128 output feature maps per layer in a single pass. We implemented the proposed architecture on a Xilinx Zynq FPGA platform and present results showing how our implementation reduces external memory transfers and compute time in five different CNNs ranging from small ones up to the widely known large VGG16 and VGG19 CNNs. Post-synthesis simulations using Mentor Modelsim in a 28nm process with a clock frequency of 500 MHz show that the VGG19 network achieves over 450 GOp/s. By exploiting sparsity, NullHop achieves an efficiency of 368%, maintains over 98% utilization of the MAC units, and achieves a power efficiency of over 3TOp/s/W in a core area of 6.3mm${}^2$. As further proof of NullHop's usability, we interfaced its FPGA implementation with a neuromorphic event camera for real time interactive demonstrations.

References in corpus (12)

• Very Deep Convolutional Networks for Large-Scale Image Recognition
• BinaryConnect: Training Deep Neural Networks with binary weights during propagations
• DoReFa-Net: Training Low Bitwidth Convolutional Neural Networks with Low Bitwidth Gradients
• Quantized Neural Networks: Training Neural Networks with Low Precision Weights and Activations
• ENet: A Deep Neural Network Architecture for Real-Time Semantic Segmentation
• Fixed Point Quantization of Deep Convolutional Networks
• Hardware-oriented Approximation of Convolutional Neural Networks
• WRPN: Wide Reduced-Precision Networks
• EIE: Efficient Inference Engine on Compressed Deep Neural Network
• Rounding Methods for Neural Networks with Low Resolution Synaptic Weights
• Delta Networks for Optimized Recurrent Network Computation
• ADaPTION: Toolbox and Benchmark for Training Convolutional Neural Networks with Reduced Numerical Precision Weights and Activation

Cited by in corpus (34)

• Event-based Vision: A Survey
• Hardware and Software Optimizations for Accelerating Deep Neural Networks: Survey of Current Trends, Challenges, and the Road Ahead
• XNOR Neural Engine: a Hardware Accelerator IP for 21.6 fJ/op Binary Neural Network Inference
• Fast convolutional neural networks on FPGAs with hls4ml
• Hardware Acceleration of Sparse and Irregular Tensor Computations of ML Models: A Survey and Insights
• A Survey on Deep Learning Hardware Accelerators for Heterogeneous HPC Platforms
• The importance of space and time in neuromorphic cognitive agents
• Spartus: A 9.4 TOp/s FPGA-based LSTM Accelerator Exploiting Spatio-Temporal Sparsity
• An Ultra-low Power TinyML System for Real-time Visual Processing at Edge
• Resource-Efficient Deep Learning: A Survey on Model-, Arithmetic-, and Implementation-Level Techniques
• MARS: Multi-macro Architecture SRAM CIM-Based Accelerator with Co-designed Compressed Neural Networks
• Flex-PE: Flexible and SIMD Multi-Precision Processing Element for AI Workloads
• NeuPart: Using Analytical Models to Drive Energy-Efficient Partitioning of CNN Computations on Cloud-Connected Mobile Clients
• Event-based vision on FPGAs -- a survey
• A Composable Dynamic Sparse Dataflow Architecture for Efficient Event-based Vision Processing on FPGA
• FlexNN: A Dataflow-aware Flexible Deep Learning Accelerator for Energy-Efficient Edge Devices
• Comprehensive SNN Compression Using ADMM Optimization and Activity Regularization
• Tensor Slicing and Optimization for Multicore NPUs
• HarDNet: A Low Memory Traffic Network
• A 1Mb mixed-precision quantized encoder for image classification and patch-based compression
• Hardware-software co-exploration with racetrack memory based in-memory computing for CNN inference in embedded systems
• Computation on Sparse Neural Networks: an Inspiration for Future Hardware
• HPIPE: Heterogeneous Layer-Pipelined and Sparse-Aware CNN Inference for FPGAs
• Invariance Measures for Neural Networks
• Always-On 674uW @ 4GOP/s Error Resilient Binary Neural Networks with Aggressive SRAM Voltage Scaling on a 22nm IoT End-Node
• Dual-side Sparse Tensor Core
• GNNIE: GNN Inference Engine with Load-balancing and Graph-Specific Caching
• Performance evaluation over HW/SW co-design SoC memory transfers for a CNN accelerator
• ASC: Adaptive Scale Feature Map Compression for Deep Neural Network
• Training for temporal sparsity in deep neural networks, application in video processing
• Data-Driven Neuromorphic DRAM-based CNN and RNN Accelerators
• A CNN Accelerator on FPGA Using Depthwise Separable Convolution
• HYDRA: Hybrid Data Multiplexing and Run-time Layer Configurable DNN Accelerator
• Transform-Based Feature Map Compression for CNN Inference