DC-NAS: Divide-and-Conquer Neural Architecture Search

Abstract

Most applications demand high-performance deep neural architectures costing limited resources. Neural architecture searching is a way of automatically exploring optimal deep neural networks in a given huge search space. However, all sub-networks are usually evaluated using the same criterion; that is, early stopping on a small proportion of the training dataset, which is an inaccurate and highly complex approach. In contrast to conventional methods, here we present a divide-and-conquer (DC) approach to effectively and efficiently search deep neural architectures. Given an arbitrary search space, we first extract feature representations of all sub-networks according to changes in parameters or output features of each layer, and then calculate the similarity between two different sampled networks based on the representations. Then, a k-means clustering is conducted to aggregate similar architectures into the same cluster, separately executing sub-network evaluation in each cluster. The best architecture in each cluster is later merged to obtain the optimal neural architecture. Experimental results conducted on several benchmarks illustrate that DC-NAS can overcome the inaccurate evaluation problem, achieving a $75.1\%$ top-1 accuracy on the ImageNet dataset, which is higher than that of state-of-the-art methods using the same search space.

References in corpus (14)

• MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applications
• Neural Architecture Search with Reinforcement Learning
• MobileNetV2: Inverted Residuals and Linear Bottlenecks
• Efficient Neural Architecture Search via Parameter Sharing
• SNAS: Stochastic Neural Architecture Search
• Random Search and Reproducibility for Neural Architecture Search
• Evaluating the Search Phase of Neural Architecture Search
• Freeze-Thaw Bayesian Optimization
• GhostNet: More Features from Cheap Operations
• Peephole: Predicting Network Performance Before Training
• CARS: Continuous Evolution for Efficient Neural Architecture Search
• ReNAS:Relativistic Evaluation of Neural Architecture Search
• Hit-Detector: Hierarchical Trinity Architecture Search for Object Detection
• A Semi-Supervised Assessor of Neural Architectures

Cited by in corpus (2)

• Weight-Sharing Neural Architecture Search: A Battle to Shrink the Optimization Gap
• Stronger NAS with Weaker Predictors