Balanced Quantization: An Effective and Efficient Approach to Quantized Neural Networks

Abstract

Quantized Neural Networks (QNNs), which use low bitwidth numbers for representing parameters and performing computations, have been proposed to reduce the computation complexity, storage size and memory usage. In QNNs, parameters and activations are uniformly quantized, such that the multiplications and additions can be accelerated by bitwise operations. However, distributions of parameters in Neural Networks are often imbalanced, such that the uniform quantization determined from extremal values may under utilize available bitwidth. In this paper, we propose a novel quantization method that can ensure the balance of distributions of quantized values. Our method first recursively partitions the parameters by percentiles into balanced bins, and then applies uniform quantization. We also introduce computationally cheaper approximations of percentiles to reduce the computation overhead introduced. Overall, our method improves the prediction accuracies of QNNs without introducing extra computation during inference, has negligible impact on training speed, and is applicable to both Convolutional Neural Networks and Recurrent Neural Networks. Experiments on standard datasets including ImageNet and Penn Treebank confirm the effectiveness of our method. On ImageNet, the top-5 error rate of our 4-bit quantized GoogLeNet model is 12.7\%, which is superior to the state-of-the-arts of QNNs.

References in corpus (21)

• Very Deep Convolutional Networks for Large-Scale Image Recognition
• Empirical Evaluation of Gated Recurrent Neural Networks on Sequence Modeling
• Estimating or Propagating Gradients Through Stochastic Neurons for Conditional Computation
• BinaryConnect: Training Deep Neural Networks with binary weights during propagations
• DoReFa-Net: Training Low Bitwidth Convolutional Neural Networks with Low Bitwidth Gradients
• Exploiting Linear Structure Within Convolutional Networks for Efficient Evaluation
• Deep Learning with Limited Numerical Precision
• Compressing Deep Convolutional Networks using Vector Quantization
• Binarized Neural Networks
• Tensorizing Neural Networks
• Speeding up Convolutional Neural Networks with Low Rank Expansions
• Convolutional neural networks with low-rank regularization
• Convolutional Neural Networks using Logarithmic Data Representation
• Hardware-oriented Approximation of Convolutional Neural Networks
• Bitwise Neural Networks
• Deep Neural Networks with Random Gaussian Weights: A Universal Classification Strategy?
• Deep neural networks are robust to weight binarization and other non-linear distortions
• An exploration of parameter redundancy in deep networks with circulant projections
• Structured Pruning of Deep Convolutional Neural Networks
• Quantized Convolutional Neural Networks for Mobile Devices
• Exploiting Local Structures with the Kronecker Layer in Convolutional Networks