Accelerating Deep Convolutional Networks using low-precision and sparsity

Abstract

We explore techniques to significantly improve the compute efficiency and performance of Deep Convolution Networks without impacting their accuracy. To improve the compute efficiency, we focus on achieving high accuracy with extremely low-precision (2-bit) weight networks, and to accelerate the execution time, we aggressively skip operations on zero-values. We achieve the highest reported accuracy of 76.6% Top-1/93% Top-5 on the Imagenet object classification challenge with low-precision network\footnote{github release of the source code coming soon} while reducing the compute requirement by ~3x compared to a full-precision network that achieves similar accuracy. Furthermore, to fully exploit the benefits of our low-precision networks, we build a deep learning accelerator core, dLAC, that can achieve up to 1 TFLOP/mm^2 equivalent for single-precision floating-point operations (~2 TFLOP/mm^2 for half-precision).

References in corpus (2)

• Very Deep Convolutional Networks for Large-Scale Image Recognition
• XNOR-Net: ImageNet Classification Using Binary Convolutional Neural Networks

Cited by in corpus (10)

• Bayesian Compression for Deep Learning
• Flexpoint: An Adaptive Numerical Format for Efficient Training of Deep Neural Networks
• Deep Convolutional Neural Network Inference with Floating-point Weights and Fixed-point Activations
• FPGA-Based CNN Inference Accelerator Synthesized from Multi-Threaded C Software
• Learning Accurate Low-Bit Deep Neural Networks with Stochastic Quantization
• Mixed Low-precision Deep Learning Inference using Dynamic Fixed Point
• Bit-Tactical: Exploiting Ineffectual Computations in Convolutional Neural Networks: Which, Why, and How
• WRPN: Training and Inference using Wide Reduced-Precision Networks
• BitNet: Bit-Regularized Deep Neural Networks
• Irregular Convolutional Neural Networks