BETANAS: BalancEd TrAining and selective drop for Neural Architecture Search

Abstract

Automatic neural architecture search techniques are becoming increasingly important in machine learning area. Especially, weight sharing methods have shown remarkable potentials on searching good network architectures with few computational resources. However, existing weight sharing methods mainly suffer limitations on searching strategies: these methods either uniformly train all network paths to convergence which introduces conflicts between branches and wastes a large amount of computation on unpromising candidates, or selectively train branches with different frequency which leads to unfair evaluation and comparison among paths. To address these issues, we propose a novel neural architecture search method with balanced training strategy to ensure fair comparisons and a selective drop mechanism to reduce conflicts among candidate paths. The experimental results show that our proposed method can achieve a leading performance of 79.0% on ImageNet under mobile settings, which outperforms other state-of-the-art methods in both accuracy and efficiency.

References in corpus (15)

• EfficientNet: Rethinking Model Scaling for Convolutional Neural Networks
• Neural Architecture Search with Reinforcement Learning
• Searching for Activation Functions
• AutoAugment: Learning Augmentation Policies from Data
• Efficient Neural Architecture Search via Parameter Sharing
• SMASH: One-Shot Model Architecture Search through HyperNetworks
• DARTS+: Improved Differentiable Architecture Search with Early Stopping
• Single Path One-Shot Neural Architecture Search with Uniform Sampling
• Evaluating the Search Phase of Neural Architecture Search
• Progressive Differentiable Architecture Search: Bridging the Depth Gap between Search and Evaluation
• FBNet: Hardware-Aware Efficient ConvNet Design via Differentiable Neural Architecture Search
• ASAP: Architecture Search, Anneal and Prune
• Partial Order Pruning: for Best Speed/Accuracy Trade-off in Neural Architecture Search
• IRLAS: Inverse Reinforcement Learning for Architecture Search
• HM-NAS: Efficient Neural Architecture Search via Hierarchical Masking

Cited by in corpus (6)

• Powering One-shot Topological NAS with Stabilized Share-parameter Proxy
• Collaboration of Experts: Achieving 80% Top-1 Accuracy on ImageNet with 100M FLOPs
• K-shot NAS: Learnable Weight-Sharing for NAS with K-shot Supernets
• Evolving Search Space for Neural Architecture Search
• Prioritized Architecture Sampling with Monto-Carlo Tree Search
• AutoBSS: An Efficient Algorithm for Block Stacking Style Search