Breaking the Curse of Space Explosion: Towards Efficient NAS with Curriculum Search

Abstract

Neural architecture search (NAS) has become an important approach to automatically find effective architectures. To cover all possible good architectures, we need to search in an extremely large search space with billions of candidate architectures. More critically, given a large search space, we may face a very challenging issue of space explosion. However, due to the limitation of computational resources, we can only sample a very small proportion of the architectures, which provides insufficient information for the training. As a result, existing methods may often produce suboptimal architectures. To alleviate this issue, we propose a curriculum search method that starts from a small search space and gradually incorporates the learned knowledge to guide the search in a large space. With the proposed search strategy, our Curriculum Neural Architecture Search (CNAS) method significantly improves the search efficiency and finds better architectures than existing NAS methods. Extensive experiments on CIFAR-10 and ImageNet demonstrate the effectiveness of the proposed method.

arXiv commentAccepted by ICML 2020

Cited by in corpus (11)

• Self-Paced Learning for Neural Machine Translation
• Sample-Efficient Neural Architecture Search by Learning Action Space
• A Generic Graph-based Neural Architecture Encoding Scheme for Predictor-based NAS
• Theory-Inspired Path-Regularized Differential Network Architecture Search
• Neural Feature Search for RGB-Infrared Person Re-Identification
• A Survey on Curriculum Learning
• Sharpness-aware Quantization for Deep Neural Networks
• Contrastive Neural Architecture Search with Neural Architecture Comparators
• Pareto-Frontier-aware Neural Architecture Generation for Diverse Budgets
• Rapid Elastic Architecture Search under Specialized Classes and Resource Constraints
• Towards Accurate and Compact Architectures via Neural Architecture Transformer