CP-NAS: Child-Parent Neural Architecture Search for Binary Neural Networks
arXiv:2005.00057
Abstract
Neural architecture search (NAS) proves to be among the best approaches for many tasks by generating an application-adaptive neural architecture, which is still challenged by high computational cost and memory consumption. At the same time, 1-bit convolutional neural networks (CNNs) with binarized weights and activations show their potential for resource-limited embedded devices. One natural approach is to use 1-bit CNNs to reduce the computation and memory cost of NAS by taking advantage of the strengths of each in a unified framework. To this end, a Child-Parent (CP) model is introduced to a differentiable NAS to search the binarized architecture (Child) under the supervision of a full-precision model (Parent). In the search stage, the Child-Parent model uses an indicator generated by the child and parent model accuracy to evaluate the performance and abandon operations with less potential. In the training stage, a kernel-level CP loss is introduced to optimize the binarized network. Extensive experiments demonstrate that the proposed CP-NAS achieves a comparable accuracy with traditional NAS on both the CIFAR and ImageNet databases. It achieves the accuracy of on CIFAR-10, on ImageNet with binarized weights and activations, and a faster search than prior arts.
arXiv comment7 pages, 6 figures
References in corpus (9)
- Distilling the Knowledge in a Neural Network
- Neural Architecture Search with Reinforcement Learning
- Improved Regularization of Convolutional Neural Networks with Cutout
- Binarized Neural Networks: Training Deep Neural Networks with Weights and Activations Constrained to +1 or -1
- DARTS: Differentiable Architecture Search
- Efficient Neural Architecture Search via Parameter Sharing
- Efficient Architecture Search by Network Transformation
- ProxylessNAS: Direct Neural Architecture Search on Target Task and Hardware
- Path-Level Network Transformation for Efficient Architecture Search