XNOR Neural Engine: a Hardware Accelerator IP for 21.6 fJ/op Binary Neural Network Inference

Abstract

Binary Neural Networks (BNNs) are promising to deliver accuracy comparable to conventional deep neural networks at a fraction of the cost in terms of memory and energy. In this paper, we introduce the XNOR Neural Engine (XNE), a fully digital configurable hardware accelerator IP for BNNs, integrated within a microcontroller unit (MCU) equipped with an autonomous I/O subsystem and hybrid SRAM / standard cell memory. The XNE is able to fully compute convolutional and dense layers in autonomy or in cooperation with the core in the MCU to realize more complex behaviors. We show post-synthesis results in 65nm and 22nm technology for the XNE IP and post-layout results in 22nm for the full MCU indicating that this system can drop the energy cost per binary operation to 21.6fJ per operation at 0.4V, and at the same time is flexible and performant enough to execute state-of-the-art BNN topologies such as ResNet-34 in less than 2.2mJ per frame at 8.9 fps.

arXiv comment11 pages, 8 figures, 2 tables, 3 listings. Accepted for presentation at CODES'18 and for publication in IEEE Transactions on Computer-Aided Design of Circuits and Systems (TCAD) as part of the ESWEEK-TCAD special issue

References in corpus (13)

• Very Deep Convolutional Networks for Large-Scale Image Recognition
• SqueezeNet: AlexNet-level accuracy with 50x fewer parameters and <0.5MB model size
• Binarized Neural Networks: Training Deep Neural Networks with Weights and Activations Constrained to +1 or -1
• BinaryConnect: Training Deep Neural Networks with binary weights during propagations
• DoReFa-Net: Training Low Bitwidth Convolutional Neural Networks with Low Bitwidth Gradients
• FINN: A Framework for Fast, Scalable Binarized Neural Network Inference
• Incremental Network Quantization: Towards Lossless CNNs with Low-Precision Weights
• Towards Accurate Binary Convolutional Neural Network
• Xception: Deep Learning with Depthwise Separable Convolutions
• NullHop: A Flexible Convolutional Neural Network Accelerator Based on Sparse Representations of Feature Maps
• An IoT Endpoint System-on-Chip for Secure and Energy-Efficient Near-Sensor Analytics
• Energy-Efficient ConvNets Through Approximate Computing
• Towards End-to-End Speech Recognition with Deep Convolutional Neural Networks

Cited by in corpus (18)

• Hardware and Software Optimizations for Accelerating Deep Neural Networks: Survey of Current Trends, Challenges, and the Road Ahead
• Hardware Approximate Techniques for Deep Neural Network Accelerators: A Survey
• PULP-NN: Accelerating Quantized Neural Networks on Parallel Ultra-Low-Power RISC-V Processors
• Synaptic metaplasticity in binarized neural networks
• TinyVers: A Tiny Versatile System-on-chip with State-Retentive eMRAM for ML Inference at the Extreme Edge
• Marsellus: A Heterogeneous RISC-V AI-IoT End-Node SoC with 2-to-8b DNN Acceleration and 30%-Boost Adaptive Body Biasing
• Optimally Scheduling CNN Convolutions for Efficient Memory Access
• Ultra-compact Binary Neural Networks for Human Activity Recognition on RISC-V Processors
• Vau da muntanialas: Energy-efficient multi-die scalable acceleration of RNN inference
• PIMBALL: Binary Neural Networks in Spintronic Memory
• RA-BNN: Constructing Robust & Accurate Binary Neural Network to Simultaneously Defend Adversarial Bit-Flip Attack and Improve Accuracy
• An All-digital 8.6-nJ/Frame 65-nm Tsetlin Machine Image Classification Accelerator
• Towards Fast and Energy-Efficient Binarized Neural Network Inference on FPGA
• Always-On 674uW @ 4GOP/s Error Resilient Binary Neural Networks with Aggressive SRAM Voltage Scaling on a 22nm IoT End-Node
• A Machine Learning Accelerator In-Memory for Energy Harvesting
• Quantum Annealing Formulation for Binary Neural Networks
• PBGen: Partial Binarization of Deconvolution-Based Generators for Edge Intelligence
• Training Multi-bit Quantized and Binarized Networks with A Learnable Symmetric Quantizer