BNAS:An Efficient Neural Architecture Search Approach Using Broad Scalable Architecture

Abstract

In this paper, we propose Broad Neural Architecture Search (BNAS) where we elaborately design broad scalable architecture dubbed Broad Convolutional Neural Network (BCNN) to solve the above issue. On one hand, the proposed broad scalable architecture has fast training speed due to its shallow topology. Moreover, we also adopt reinforcement learning and parameter sharing used in ENAS as the optimization strategy of BNAS. Hence, the proposed approach can achieve higher search efficiency. On the other hand, the broad scalable architecture extracts multi-scale features and enhancement representations, and feeds them into global average pooling layer to yield more reasonable and comprehensive representations. Therefore, the performance of broad scalable architecture can be promised. In particular, we also develop two variants for BNAS who modify the topology of BCNN. In order to verify the effectiveness of BNAS, several experiments are performed and experimental results show that 1) BNAS delivers 0.19 days which is 2.37x less expensive than ENAS who ranks the best in reinforcement learning-based NAS approaches, 2) compared with small-size (0.5 millions parameters) and medium-size (1.1 millions parameters) models, the architecture learned by BNAS obtains state-of-the-art performance (3.58% and 3.24% test error) on CIFAR-10, 3) the learned architecture achieves 25.3% top-1 error on ImageNet just using 3.9 millions parameters.

arXiv comment15 pages, 12 figures, 5 tables

References in corpus (12)

• MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applications
• Neural Architecture Search with Reinforcement Learning
• Improved Regularization of Convolutional Neural Networks with Cutout
• SGDR: Stochastic Gradient Descent with Warm Restarts
• DARTS: Differentiable Architecture Search
• ShuffleNet: An Extremely Efficient Convolutional Neural Network for Mobile Devices
• Efficient Neural Architecture Search via Parameter Sharing
• PC-DARTS: Partial Channel Connections for Memory-Efficient Architecture Search
• SMASH: One-Shot Model Architecture Search through HyperNetworks
• Progressive Differentiable Architecture Search: Bridging the Depth Gap between Search and Evaluation
• MANAS: Multi-Agent Neural Architecture Search
• ModuleNet: Knowledge-inherited Neural Architecture Search

Cited by in corpus (4)

• Weight-Sharing Neural Architecture Search: A Battle to Shrink the Optimization Gap
• Exploring the Intersection between Neural Architecture Search and Continual Learning
• ABCP: Automatic Block-wise and Channel-wise Network Pruning via Joint Search
• BNAS-v2: Memory-efficient and Performance-collapse-prevented Broad Neural Architecture Search