Ristretto: Hardware-Oriented Approximation of Convolutional Neural Networks

Abstract

Convolutional neural networks (CNN) have achieved major breakthroughs in recent years. Their performance in computer vision have matched and in some areas even surpassed human capabilities. Deep neural networks can capture complex non-linear features; however this ability comes at the cost of high computational and memory requirements. State-of-art networks require billions of arithmetic operations and millions of parameters. To enable embedded devices such as smartphones, Google glasses and monitoring cameras with the astonishing power of deep learning, dedicated hardware accelerators can be used to decrease both execution time and power consumption. In applications where fast connection to the cloud is not guaranteed or where privacy is important, computation needs to be done locally. Many hardware accelerators for deep neural networks have been proposed recently. A first important step of accelerator design is hardware-oriented approximation of deep networks, which enables energy-efficient inference. We present Ristretto, a fast and automated framework for CNN approximation. Ristretto simulates the hardware arithmetic of a custom hardware accelerator. The framework reduces the bit-width of network parameters and outputs of resource-intense layers, which reduces the chip area for multiplication units significantly. Alternatively, Ristretto can remove the need for multipliers altogether, resulting in an adder-only arithmetic. The tool fine-tunes trimmed networks to achieve high classification accuracy. Since training of deep neural networks can be time-consuming, Ristretto uses highly optimized routines which run on the GPU. This enables fast compression of any given network. Given a maximum tolerance of 1%, Ristretto can successfully condense CaffeNet and SqueezeNet to 8-bit. The code for Ristretto is available.

arXiv commentMaster's Thesis, University of California, Davis; 73 pages and 29 figures

References in corpus (11)

• SqueezeNet: AlexNet-level accuracy with 50x fewer parameters and <0.5MB model size
• Deep Residual Learning for Image Recognition
• Binarized Neural Networks: Training Deep Neural Networks with Weights and Activations Constrained to +1 or -1
• BinaryConnect: Training Deep Neural Networks with binary weights during propagations
• Exploiting Linear Structure Within Convolutional Networks for Efficient Evaluation
• Deep Learning with Limited Numerical Precision
• cuDNN: Efficient Primitives for Deep Learning
• Fast Training of Convolutional Networks through FFTs
• XNOR-Net: ImageNet Classification Using Binary Convolutional Neural Networks
• Hardware-oriented Approximation of Convolutional Neural Networks
• Resiliency of Deep Neural Networks under Quantization

Cited by in corpus (20)

• Bayesian Compression for Deep Learning
• Hardware Acceleration of Sparse and Irregular Tensor Computations of ML Models: A Survey and Insights
• Deep Convolutional Neural Network Inference with Floating-point Weights and Fixed-point Activations
• Performance-Efficiency Trade-off of Low-Precision Numerical Formats in Deep Neural Networks
• Quantization for Rapid Deployment of Deep Neural Networks
• Exploration of Low Numeric Precision Deep Learning Inference Using Intel FPGAs
• Optimally Scheduling CNN Convolutions for Efficient Memory Access
• Exploring the Design Space of Deep Convolutional Neural Networks at Large Scale
• Automated Design Space Exploration for optimised Deployment of DNN on Arm Cortex-A CPUs
• Gradient ${\ell }_1$ Regularization for Quantization Robustness
• Full deep neural network training on a pruned weight budget
• Convolutional neural network compression for natural language processing
• Understanding the Impact of Precision Quantization on the Accuracy and Energy of Neural Networks
• Scalar Arithmetic Multiple Data: Customizable Precision for Deep Neural Networks
• GDRQ: Group-based Distribution Reshaping for Quantization
• Stacked Filters Stationary Flow For Hardware-Oriented Acceleration Of Deep Convolutional Neural Networks
• Semi-Relaxed Quantization with DropBits: Training Low-Bit Neural Networks via Bit-wise Regularization
• FPGA Implementations of 3D-SIMD Processor Architecture for Deep Neural Networks Using Relative Indexed Compressed Sparse Filter Encoding Format and Stacked Filters Stationary Flow
• Exploring Bit-Slice Sparsity in Deep Neural Networks for Efficient ReRAM-Based Deployment
• When deep learning models on GPU can be accelerated by taking advantage of unstructured sparsity