Performance Guaranteed Network Acceleration via High-Order Residual Quantization

Abstract

Input binarization has shown to be an effective way for network acceleration. However, previous binarization scheme could be regarded as simple pixel-wise thresholding operations (i.e., order-one approximation) and suffers a big accuracy loss. In this paper, we propose a highorder binarization scheme, which achieves more accurate approximation while still possesses the advantage of binary operation. In particular, the proposed scheme recursively performs residual quantization and yields a series of binary input images with decreasing magnitude scales. Accordingly, we propose high-order binary filtering and gradient propagation operations for both forward and backward computations. Theoretical analysis shows approximation error guarantee property of proposed method. Extensive experimental results demonstrate that the proposed scheme yields great recognition accuracy while being accelerated.

9 pages, 8 figures, Proceeding of IEEE International Conference on Computer Vision 2017

References in corpus (11)

• Adam: A Method for Stochastic Optimization
• Very Deep Convolutional Networks for Large-Scale Image Recognition
• Binarized Neural Networks: Training Deep Neural Networks with Weights and Activations Constrained to +1 or -1
• BinaryConnect: Training Deep Neural Networks with binary weights during propagations
• Exploiting Linear Structure Within Convolutional Networks for Efficient Evaluation
• Predicting Parameters in Deep Learning
• Speeding up Convolutional Neural Networks with Low Rank Expansions
• Spatially-sparse convolutional neural networks
• Bitwise Neural Networks
• Memory Bounded Deep Convolutional Networks
• Training Binary Multilayer Neural Networks for Image Classification using Expectation Backpropagation

Cited by in corpus (7)

• Searching for Low-Bit Weights in Quantized Neural Networks
• Structured Binary Neural Networks for Accurate Image Classification and Semantic Segmentation
• Learning Effective Binary Visual Representations with Deep Networks
• Recurrent Residual Module for Fast Inference in Videos
• Direct Quantization for Training Highly Accurate Low Bit-width Deep Neural Networks
• Cross-filter compression for CNN inference acceleration
• Cluster Regularized Quantization for Deep Networks Compression