A Precision-Scalable RISC-V DNN Processor with On-Device Learning Capability at the Extreme Edge

Abstract

Extreme edge platforms, such as in-vehicle smart devices, require efficient deployment of quantized deep neural networks (DNNs) to enable intelligent applications with limited amounts of energy, memory, and computing resources. However, many edge devices struggle to boost inference throughput of various quantized DNNs due to the varying quantization levels, and these devices lack floating-point (FP) support for on-device learning, which prevents them from improving model accuracy while ensuring data privacy. To tackle the challenges above, we propose a precision-scalable RISC-V DNN processor with on-device learning capability. It facilitates diverse precision levels of fixed-point DNN inference, spanning from 2-bit to 16-bit, and enhances on-device learning through improved support with FP16 operations. Moreover, we employ multiple methods such as FP16 multiplier reuse and multi-precision integer multiplier reuse, along with balanced mapping of FPGA resources, to significantly improve hardware resource utilization. Experimental results on the Xilinx ZCU102 FPGA show that our processor significantly improves inference throughput by 1.6$\sim$14.6$\times$ and energy efficiency by 1.1$\sim$14.6$\times$ across various DNNs, compared to the prior art, XpulpNN. Additionally, our processor achieves a 16.5$\times$ higher FP throughput for on-device learning.

arXiv commentTo appear in the 29th Asia and South Pacific Design Automation Conference (ASP-DAC 2024)

References in corpus (3)

• Dustin: A 16-Cores Parallel Ultra-Low-Power Cluster with 2b-to-32b Fully Flexible Bit-Precision and Vector Lockstep Execution Mode
• BEBERT: Efficient and Robust Binary Ensemble BERT
• BARVINN: Arbitrary Precision DNN Accelerator Controlled by a RISC-V CPU

Cited by in corpus (7)

• Anda: Unlocking Efficient LLM Inference with a Variable-Length Grouped Activation Data Format
• A Scalable RISC-V Vector Processor Enabling Efficient Multi-Precision DNN Inference
• SPEED: A Scalable RISC-V Vector Processor Enabling Efficient Multi-Precision DNN Inference
• Co-Designing Binarized Transformer and Hardware Accelerator for Efficient End-to-End Edge Deployment
• Efficient Precision-Scalable Hardware for Microscaling (MX) Processing in Robotics Learning
• Precision-Scalable Microscaling Datapaths with Optimized Reduction Tree for Efficient NPU Integration
• A Scheduling Framework for Efficient MoE Inference on Edge GPU-NDP Systems