Projection Convolutional Neural Networks for 1-bit CNNs via Discrete Back Propagation

Abstract

The advancement of deep convolutional neural networks (DCNNs) has driven significant improvement in the accuracy of recognition systems for many computer vision tasks. However, their practical applications are often restricted in resource-constrained environments. In this paper, we introduce projection convolutional neural networks (PCNNs) with a discrete back propagation via projection (DBPP) to improve the performance of binarized neural networks (BNNs). The contributions of our paper include: 1) for the first time, the projection function is exploited to efficiently solve the discrete back propagation problem, which leads to a new highly compressed CNNs (termed PCNNs); 2) by exploiting multiple projections, we learn a set of diverse quantized kernels that compress the full-precision kernels in a more efficient way than those proposed previously; 3) PCNNs achieve the best classification performance compared to other state-of-the-art BNNs on the ImageNet and CIFAR datasets.

References in corpus (14)

• Very Deep Convolutional Networks for Large-Scale Image Recognition
• MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applications
• Binarized Neural Networks: Training Deep Neural Networks with Weights and Activations Constrained to +1 or -1
• Wide Residual Networks
• BinaryConnect: Training Deep Neural Networks with binary weights during propagations
• DoReFa-Net: Training Low Bitwidth Convolutional Neural Networks with Low Bitwidth Gradients
• Exploiting Linear Structure Within Convolutional Networks for Efficient Evaluation
• Quantized Neural Networks: Training Neural Networks with Low Precision Weights and Activations
• Towards Accurate Binary Convolutional Neural Network
• Convolutional neural networks with low-rank regularization
• Channel Pruning for Accelerating Very Deep Neural Networks
• Training wide residual networks for deployment using a single bit for each weight
• Oriented Response Networks
• Bi-Real Net: Enhancing the Performance of 1-bit CNNs With Improved Representational Capability and Advanced Training Algorithm

Cited by in corpus (2)

• Searching for Low-Bit Weights in Quantized Neural Networks
• Reducing the Computational Cost of Deep Generative Models with Binary Neural Networks