CARS: Continuous Evolution for Efficient Neural Architecture Search

Abstract

Searching techniques in most of existing neural architecture search (NAS) algorithms are mainly dominated by differentiable methods for the efficiency reason. In contrast, we develop an efficient continuous evolutionary approach for searching neural networks. Architectures in the population that share parameters within one SuperNet in the latest generation will be tuned over the training dataset with a few epochs. The searching in the next evolution generation will directly inherit both the SuperNet and the population, which accelerates the optimal network generation. The non-dominated sorting strategy is further applied to preserve only results on the Pareto front for accurately updating the SuperNet. Several neural networks with different model sizes and performances will be produced after the continuous search with only 0.4 GPU days. As a result, our framework provides a series of networks with the number of parameters ranging from 3.7M to 5.1M under mobile settings. These networks surpass those produced by the state-of-the-art methods on the benchmark ImageNet dataset.

arXiv commentTo be published in CVPR2020

References in corpus (19)

• Neural Architecture Search with Reinforcement Learning
• Accurate, Large Minibatch SGD: Training ImageNet in 1 Hour
• DARTS: Differentiable Architecture Search
• Binarized Neural Networks
• Efficient Neural Architecture Search via Parameter Sharing
• Efficient Architecture Search by Network Transformation
• NAS-Bench-101: Towards Reproducible Neural Architecture Search
• ShuffleNet V2: Practical Guidelines for Efficient CNN Architecture Design
• Single Path One-Shot Neural Architecture Search with Uniform Sampling
• MONAS: Multi-Objective Neural Architecture Search using Reinforcement Learning
• AutoSlim: Towards One-Shot Architecture Search for Channel Numbers
• GhostNet: More Features from Cheap Operations
• Learning Sparse Networks Using Targeted Dropout
• Probabilistic Neural Architecture Search
• Towards Optimal Structured CNN Pruning via Generative Adversarial Learning
• Positive-Unlabeled Compression on the Cloud
• Progressive DARTS: Bridging the Optimization Gap for NAS in the Wild
• DPP-Net: Device-aware Progressive Search for Pareto-optimal Neural Architectures
• Attribute Aware Pooling for Pedestrian Attribute Recognition

Cited by in corpus (7)

• A Survey on Evolutionary Neural Architecture Search
• GhostNet: More Features from Cheap Operations
• ReNAS:Relativistic Evaluation of Neural Architecture Search
• A Semi-Supervised Assessor of Neural Architectures
• MiLeNAS: Efficient Neural Architecture Search via Mixed-Level Reformulation
• DC-NAS: Divide-and-Conquer Neural Architecture Search
• Differentiable Neural Architecture Learning for Efficient Neural Network Design