Structured Binary Neural Networks for Accurate Image Classification and Semantic Segmentation
arXiv:1811.10413
Abstract
In this paper, we propose to train convolutional neural networks (CNNs) with both binarized weights and activations, leading to quantized models specifically} for mobile devices with limited power capacity and computation resources. Previous works on quantizing CNNs seek to approximate the floating-point information using a set of discrete values, which we call value approximation, but typically assume the same architecture as the full-precision networks. In this paper, however, we take a novel 'structure approximation' view for quantization---it is very likely that a different architecture may be better for best performance. In particular, we propose a `network decomposition' strategy, named \textbf{Group-Net}, in which we divide the network into groups. In this way, each full-precision group can be effectively reconstructed by aggregating a set of homogeneous binary branches. In addition, we learn effective connections among groups to improve the representational capability. Moreover, the proposed Group-Net shows strong generalization to other tasks. For instance, we extend Group-Net for highly accurate semantic segmentation by embedding rich context into the binary structure. Experiments on both classification and semantic segmentation tasks demonstrate the superior performance of the proposed methods over various popular architectures. In particular, we outperform the previous best binary neural networks in terms of accuracy and major computation savings.
arXiv commentarXiv admin note: text overlap with arXiv:1808.02631
References in corpus (36)
- Adam: A Method for Stochastic Optimization
- Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks
- MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applications
- Rethinking Atrous Convolution for Semantic Image Segmentation
- SqueezeNet: AlexNet-level accuracy with 50x fewer parameters and <0.5MB model size
- Deep Residual Learning for Image Recognition
- Neural Architecture Search with Reinforcement Learning
- MobileNetV2: Inverted Residuals and Linear Bottlenecks
- Estimating or Propagating Gradients Through Stochastic Neurons for Conditional Computation
- MXNet: A Flexible and Efficient Machine Learning Library for Heterogeneous Distributed Systems
- BinaryConnect: Training Deep Neural Networks with binary weights during propagations
- DoReFa-Net: Training Low Bitwidth Convolutional Neural Networks with Low Bitwidth Gradients
- Quantized Neural Networks: Training Neural Networks with Low Precision Weights and Activations
- Going Deeper with Convolutions
- Binarized Neural Networks
- Fully Convolutional Networks for Semantic Segmentation
- Trained Ternary Quantization
- Rethinking the Inception Architecture for Computer Vision
- Incremental Network Quantization: Towards Lossless CNNs with Low-Precision Weights
- Towards Accurate Binary Convolutional Neural Network
- Aggregated Residual Transformations for Deep Neural Networks
- Xception: Deep Learning with Depthwise Separable Convolutions
- Channel Pruning for Accelerating Very Deep Neural Networks
- Model compression via distillation and quantization
- Loss-aware Binarization of Deep Networks
- Progressive Neural Architecture Search
- Apprentice: Using Knowledge Distillation Techniques To Improve Low-Precision Network Accuracy
- Loss-aware Weight Quantization of Deep Networks
- Deep Learning with Low Precision by Half-wave Gaussian Quantization
- LQ-Nets: Learned Quantization for Highly Accurate and Compact Deep Neural Networks
- Heterogeneous Bitwidth Binarization in Convolutional Neural Networks
- Bi-Real Net: Enhancing the Performance of 1-bit CNNs With Improved Representational Capability and Advanced Training Algorithm
- Towards Effective Low-bitwidth Convolutional Neural Networks
- Performance Guaranteed Network Acceleration via High-Order Residual Quantization
- Learning to Train a Binary Neural Network
- Network Sketching: Exploiting Binary Structure in Deep CNNs