Towards Accurate Binary Convolutional Neural Network

Abstract

We introduce a novel scheme to train binary convolutional neural networks (CNNs) -- CNNs with weights and activations constrained to {-1,+1} at run-time. It has been known that using binary weights and activations drastically reduce memory size and accesses, and can replace arithmetic operations with more efficient bitwise operations, leading to much faster test-time inference and lower power consumption. However, previous works on binarizing CNNs usually result in severe prediction accuracy degradation. In this paper, we address this issue with two major innovations: (1) approximating full-precision weights with the linear combination of multiple binary weight bases; (2) employing multiple binary activations to alleviate information loss. The implementation of the resulting binary CNN, denoted as ABC-Net, is shown to achieve much closer performance to its full-precision counterpart, and even reach the comparable prediction accuracy on ImageNet and forest trail datasets, given adequate binary weight bases and activations.

Cited by in corpus (7)

• Deep Neural Network Approximation for Custom Hardware: Where We've Been, Where We're Going
• Regularizing Activation Distribution for Training Binarized Deep Networks
• Projection Convolutional Neural Networks for 1-bit CNNs via Discrete Back Propagation
• NIPS - Not Even Wrong? A Systematic Review of Empirically Complete Demonstrations of Algorithmic Effectiveness in the Machine Learning and Artificial Intelligence Literature
• Matrix and tensor decompositions for training binary neural networks
• Modulated binary cliquenet
• Multi-Precision Quantized Neural Networks via Encoding Decomposition of -1 and +1