Binarized Neural Architecture Search for Efficient Object Recognition
arXiv:2009.04247
Abstract
Traditional neural architecture search (NAS) has a significant impact in computer vision by automatically designing network architectures for various tasks. In this paper, binarized neural architecture search (BNAS), with a search space of binarized convolutions, is introduced to produce extremely compressed models to reduce huge computational cost on embedded devices for edge computing. The BNAS calculation is more challenging than NAS due to the learning inefficiency caused by optimization requirements and the huge architecture space, and the performance loss when handling the wild data in various computing applications. To address these issues, we introduce operation space reduction and channel sampling into BNAS to significantly reduce the cost of searching. This is accomplished through a performance-based strategy that is robust to wild data, which is further used to abandon less potential operations. Furthermore, we introduce the Upper Confidence Bound (UCB) to solve 1-bit BNAS. Two optimization methods for binarized neural networks are used to validate the effectiveness of our BNAS. Extensive experiments demonstrate that the proposed BNAS achieves a comparable performance to NAS on both CIFAR and ImageNet databases. An accuracy of vs. is achieved on the CIFAR-10 dataset, but with a significantly compressed model, and a faster search than the state-of-the-art PC-DARTS. On the wild face recognition task, our binarized models achieve a performance similar to their corresponding full-precision models.
arXiv admin note: substantial text overlap with arXiv:1911.10862
References in corpus (19)
- Very Deep Convolutional Networks for Large-Scale Image Recognition
- Distilling the Knowledge in a Neural Network
- MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applications
- Neural Architecture Search with Reinforcement Learning
- Improved Regularization of Convolutional Neural Networks with Cutout
- Binarized Neural Networks: Training Deep Neural Networks with Weights and Activations Constrained to +1 or -1
- DoReFa-Net: Training Low Bitwidth Convolutional Neural Networks with Low Bitwidth Gradients
- Learning Face Representation from Scratch
- Convergence of Edge Computing and Deep Learning: A Comprehensive Survey
- DARTS: Differentiable Architecture Search
- Ternary Weight Networks
- Efficient Neural Architecture Search via Parameter Sharing
- Efficient Architecture Search by Network Transformation
- SNAS: Stochastic Neural Architecture Search
- NAS-Bench-101: Towards Reproducible Neural Architecture Search
- Path-Level Network Transformation for Efficient Architecture Search
- Extremely Low Bit Neural Network: Squeeze the Last Bit Out with ADMM
- Training wide residual networks for deployment using a single bit for each weight
- Explore no more: Improved high-probability regret bounds for non-stochastic bandits