PULP-NN: Accelerating Quantized Neural Networks on Parallel Ultra-Low-Power RISC-V Processors

Abstract

We present PULP-NN, an optimized computing library for a parallel ultra-low-power tightly coupled cluster of RISC-V processors. The key innovation in PULP-NN is a set of kernels for Quantized Neural Network (QNN) inference, targeting byte and sub-byte data types, down to INT-1, tuned for the recent trend toward aggressive quantization in deep neural network inference. The proposed library exploits both the digital signal processing (DSP) extensions available in the PULP RISC-V processors and the cluster's parallelism, achieving up to 15.5 MACs/cycle on INT-8 and improving performance by up to 63x with respect to a sequential implementation on a single RISC-V core implementing the baseline RV32IMC ISA. Using PULP-NN, a CIFAR-10 network on an octa-core cluster runs in 30x and 19.6x less clock cycles than the current state-of-the-art ARM CMSIS-NN library, running on STM32L4 and STM32H7 MCUs, respectively. The proposed library, when running on GAP-8 processor, outperforms by 36.8x and by 7.45x the execution on energy efficient MCUs such as STM32L4 and high-end MCUs such as STM32H7 respectively, when operating at the maximum frequency. The energy efficiency on GAP-8 is 14.1x higher than STM32L4 and 39.5x higher than STM32H7, at the maximum efficiency operating point.

13 pages, 11 figures, 2 tables

References in corpus (6)

• Quantized Neural Networks: Training Neural Networks with Low Precision Weights and Activations
• A 64mW DNN-based Visual Navigation Engine for Autonomous Nano-Drones
• Origami: A 803 GOp/s/W Convolutional Network Accelerator
• XNOR Neural Engine: a Hardware Accelerator IP for 21.6 fJ/op Binary Neural Network Inference
• An IoT Endpoint System-on-Chip for Secure and Energy-Efficient Near-Sensor Analytics
• NEURAghe: Exploiting CPU-FPGA Synergies for Efficient and Flexible CNN Inference Acceleration on Zynq SoCs

Cited by in corpus (17)

• Machine Learning for Microcontroller-Class Hardware: A Review
• DORY: Automatic End-to-End Deployment of Real-World DNNs on Low-Cost IoT MCUs
• pLUTo: Enabling Massively Parallel Computation in DRAM via Lookup Tables
• Sound Event Detection with Binary Neural Networks on Tightly Power-Constrained IoT Devices
• DARKSIDE: A Heterogeneous RISC-V Compute Cluster for Extreme-Edge On-Chip DNN Inference and Training
• Improving Autonomous Nano-drones Performance via Automated End-to-End Optimization and Deployment of DNNs
• Dustin: A 16-Cores Parallel Ultra-Low-Power Cluster with 2b-to-32b Fully Flexible Bit-Precision and Vector Lockstep Execution Mode
• Enabling Mixed-Precision Quantized Neural Networks in Extreme-Edge Devices
• Power-Performance Characterization of TinyML Systems
• TCN Mapping Optimization for Ultra-Low Power Time-Series Edge Inference
• Exploring Automatic Gym Workouts Recognition Locally On Wearable Resource-Constrained Devices
• Shifting Capsule Networks from the Cloud to the Deep Edge
• Q-EEGNet: an Energy-Efficient 8-bit Quantized Parallel EEGNet Implementation for Edge Motor-Imagery Brain--Machine Interfaces
• Energy-Efficient Adaptive Machine Learning on IoT End-Nodes With Class-Dependent Confidence
• Model-Driven Dataset Generation for Data-Driven Battery SOH Models
• End-to-end 100-TOPS/W Inference With Analog In-Memory Computing: Are We There Yet?
• A flexible framework for early power and timing comparison of time-multiplexed CGRA kernel executions