Quantization of Deep Neural Networks for Accumulator-constrained Processors

Abstract

We introduce an Artificial Neural Network (ANN) quantization methodology for platforms without wide accumulation registers. This enables fixed-point model deployment on embedded compute platforms that are not specifically designed for large kernel computations (i.e. accumulator-constrained processors). We formulate the quantization problem as a function of accumulator size, and aim to maximize the model accuracy by maximizing bit width of input data and weights. To reduce the number of configurations to consider, only solutions that fully utilize the available accumulator bits are being tested. We demonstrate that 16-bit accumulators are able to obtain a classification accuracy within 1\% of the floating-point baselines on the CIFAR-10 and ILSVRC2012 image classification benchmarks. Additionally, a near-optimal $2\times$ speedup is obtained on an ARM processor, by exploiting 16-bit accumulators for image classification on the All-CNN-C and AlexNet networks.

arXiv comment20 pages, 13 figures

References in corpus (12)

• Striving for Simplicity: The All Convolutional Net
• DoReFa-Net: Training Low Bitwidth Convolutional Neural Networks with Low Bitwidth Gradients
• Quantized Neural Networks: Training Neural Networks with Low Precision Weights and Activations
• Deep Learning with Limited Numerical Precision
• One weird trick for parallelizing convolutional neural networks
• Fixed Point Quantization of Deep Convolutional Networks
• Training deep neural networks with low precision multiplications
• Hardware-oriented Approximation of Convolutional Neural Networks
• Flexpoint: An Adaptive Numerical Format for Efficient Training of Deep Neural Networks
• Mixed Precision Training of Convolutional Neural Networks using Integer Operations
• Deep Convolutional Neural Network Inference with Floating-point Weights and Fixed-point Activations
• Estimating or Propagating Gradients Through Stochastic Neurons

Cited by in corpus (1)

• WrapNet: Neural Net Inference with Ultra-Low-Resolution Arithmetic