Modulated binary cliquenet

Abstract

Although Convolutional Neural Networks (CNNs) achieve effectiveness in various computer vision tasks, the significant requirement of storage of such networks hinders the deployment on computationally limited devices. In this paper, we propose a new compact and portable deep learning network named Modulated Binary Cliquenet (MBCliqueNet) aiming to improve the portability of CNNs based on binarized filters while achieving comparable performance with the full-precision CNNs like Resnet. In MBCliqueNet, we introduce a novel modulated operation to approximate the unbinarized filters and gives an initialization method to speed up its convergence. We reduce the extra parameters caused by modulated operation with parameters sharing. As a result, the proposed MBCliqueNet can reduce the required storage space of convolutional filters by a factor of at least 32, in contrast to the full-precision model, and achieve better performance than other state-of-the-art binarized models. More importantly, our model compares even better with some full-precision models like Resnet on the dataset we used.

arXiv comment5 pages, 3 figures, 2 tables

References in corpus (7)

• Binarized Neural Networks: Training Deep Neural Networks with Weights and Activations Constrained to +1 or -1
• SGDR: Stochastic Gradient Descent with Warm Restarts
• Quantized Neural Networks: Training Neural Networks with Low Precision Weights and Activations
• Towards Accurate Binary Convolutional Neural Network
• XNOR-Net: ImageNet Classification Using Binary Convolutional Neural Networks
• Deep Learning with Low Precision by Half-wave Gaussian Quantization
• Training wide residual networks for deployment using a single bit for each weight