BlockQNN: Efficient Block-wise Neural Network Architecture Generation

Abstract

Convolutional neural networks have gained a remarkable success in computer vision. However, most usable network architectures are hand-crafted and usually require expertise and elaborate design. In this paper, we provide a block-wise network generation pipeline called BlockQNN which automatically builds high-performance networks using the Q-Learning paradigm with epsilon-greedy exploration strategy. The optimal network block is constructed by the learning agent which is trained to choose component layers sequentially. We stack the block to construct the whole auto-generated network. To accelerate the generation process, we also propose a distributed asynchronous framework and an early stop strategy. The block-wise generation brings unique advantages: (1) it yields state-of-the-art results in comparison to the hand-crafted networks on image classification, particularly, the best network generated by BlockQNN achieves 2.35% top-1 error rate on CIFAR-10. (2) it offers tremendous reduction of the search space in designing networks, spending only 3 days with 32 GPUs. A faster version can yield a comparable result with only 1 GPU in 20 hours. (3) it has strong generalizability in that the network built on CIFAR also performs well on the larger-scale dataset. The best network achieves very competitive accuracy of 82.0% top-1 and 96.0% top-5 on ImageNet.

arXiv comment14 pages, 18 figures

References in corpus (14)

• Adam: A Method for Stochastic Optimization
• Very Deep Convolutional Networks for Large-Scale Image Recognition
• Neural Architecture Search with Reinforcement Learning
• Improved Regularization of Convolutional Neural Networks with Cutout
• Wide Residual Networks
• SGDR: Stochastic Gradient Descent with Warm Restarts
• SMASH: One-Shot Model Architecture Search through HyperNetworks
• Xception: Deep Learning with Depthwise Separable Convolutions
• Hierarchical Representations for Efficient Architecture Search
• Convolutional Neural Fabrics
• Progressive Neural Architecture Search
• Path-Level Network Transformation for Efficient Architecture Search
• A Genetic Programming Approach to Designing Convolutional Neural Network Architectures
• Dual Path Networks

Cited by in corpus (3)

• Single Path One-Shot Neural Architecture Search with Uniform Sampling
• Neural Architecture Search for Deep Face Recognition
• MiLeNAS: Efficient Neural Architecture Search via Mixed-Level Reformulation