Composite Binary Decomposition Networks
arXiv:1811.06668
Abstract
Binary neural networks have great resource and computing efficiency, while suffer from long training procedure and non-negligible accuracy drops, when comparing to the full-precision counterparts. In this paper, we propose the composite binary decomposition networks (CBDNet), which first compose real-valued tensor of each layer with a limited number of binary tensors, and then decompose some conditioned binary tensors into two low-rank binary tensors, so that the number of parameters and operations are greatly reduced comparing to the original ones. Experiments demonstrate the effectiveness of the proposed method, as CBDNet can approximate image classification network ResNet-18 using 5.25 bits, VGG-16 using 5.47 bits, DenseNet-121 using 5.72 bits, object detection networks SSD300 using 4.38 bits, and semantic segmentation networks SegNet using 5.18 bits, all with minor accuracy drops.
References in corpus (18)
- Very Deep Convolutional Networks for Large-Scale Image Recognition
- Distilling the Knowledge in a Neural Network
- MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applications
- Neural Architecture Search with Reinforcement Learning
- BinaryConnect: Training Deep Neural Networks with binary weights during propagations
- DoReFa-Net: Training Low Bitwidth Convolutional Neural Networks with Low Bitwidth Gradients
- Exploiting Linear Structure Within Convolutional Networks for Efficient Evaluation
- The Cityscapes Dataset for Semantic Urban Scene Understanding
- Compressing Neural Networks with the Hashing Trick
- Speeding up Convolutional Neural Networks with Low Rank Expansions
- Learning Structured Sparsity in Deep Neural Networks
- Towards Accurate Binary Convolutional Neural Network
- Loss-aware Binarization of Deep Networks
- Progressive Neural Architecture Search
- DSOD: Learning Deeply Supervised Object Detectors from Scratch
- Network Decoupling: From Regular to Depthwise Separable Convolutions
- Learning Accurate Low-Bit Deep Neural Networks with Stochastic Quantization
- Quantizing Convolutional Neural Networks for Low-Power High-Throughput Inference Engines