NAS-Bench-1Shot1: Benchmarking and Dissecting One-shot Neural Architecture Search
arXiv:2001.10422
Abstract
One-shot neural architecture search (NAS) has played a crucial role in making NAS methods computationally feasible in practice. Nevertheless, there is still a lack of understanding on how these weight-sharing algorithms exactly work due to the many factors controlling the dynamics of the process. In order to allow a scientific study of these components, we introduce a general framework for one-shot NAS that can be instantiated to many recently-introduced variants and introduce a general benchmarking framework that draws on the recent large-scale tabular benchmark NAS-Bench-101 for cheap anytime evaluations of one-shot NAS methods. To showcase the framework, we compare several state-of-the-art one-shot NAS methods, examine how sensitive they are to their hyperparameters and how they can be improved by tuning their hyperparameters, and compare their performance to that of blackbox optimizers for NAS-Bench-101.
arXiv commentIn: International Conference on Learning Representations (ICLR 2020); 19 pages, 17 figures
References in corpus (11)
- Neural Architecture Search with Reinforcement Learning
- Categorical Reparameterization with Gumbel-Softmax
- Improved Regularization of Convolutional Neural Networks with Cutout
- DARTS: Differentiable Architecture Search
- Large-Scale Evolution of Image Classifiers
- Neural Architecture Optimization
- Understanding and Robustifying Differentiable Architecture Search
- Random Search and Reproducibility for Neural Architecture Search
- Convolutional Neural Fabrics
- Best Practices for Scientific Research on Neural Architecture Search
- Probabilistic Neural Architecture Search
Cited by in corpus (37)
- NAS-FAS: Static-Dynamic Central Difference Network Search for Face Anti-Spoofing
- Neuroevolution in Deep Neural Networks: Current Trends and Future Challenges
- NATS-Bench: Benchmarking NAS Algorithms for Architecture Topology and Size
- SMAC3: A Versatile Bayesian Optimization Package for Hyperparameter Optimization
- NAS-Bench-201: Extending the Scope of Reproducible Neural Architecture Search
- Stabilizing Differentiable Architecture Search via Perturbation-based Regularization
- EEEA-Net: An Early Exit Evolutionary Neural Architecture Search
- A Comprehensive Survey on Hardware-Aware Neural Architecture Search
- Towards Green Automated Machine Learning: Status Quo and Future Directions
- EPE-NAS: Efficient Performance Estimation Without Training for Neural Architecture Search
- Neural Architecture Search without Training
- Contrastive Self-supervised Neural Architecture Search
- Does Unsupervised Architecture Representation Learning Help Neural Architecture Search?
- Evaluating Efficient Performance Estimators of Neural Architectures
- Weight-Sharing Neural Architecture Search: A Battle to Shrink the Optimization Gap
- How to Train Your Super-Net: An Analysis of Training Heuristics in Weight-Sharing NAS
- Differential Evolution for Neural Architecture Search
- How Powerful are Performance Predictors in Neural Architecture Search?
- Neural Architecture Generator Optimization
- Speedy Performance Estimation for Neural Architecture Search
- To Share or Not To Share: A Comprehensive Appraisal of Weight-Sharing
- Auto-PyTorch Tabular: Multi-Fidelity MetaLearning for Efficient and Robust AutoDL
- Few-shot Neural Architecture Search
- NASI: Label- and Data-agnostic Neural Architecture Search at Initialization
- DrNAS: Dirichlet Neural Architecture Search
- The Nonlinearity Coefficient - A Practical Guide to Neural Architecture Design
- Bag of Tricks for Neural Architecture Search
- Fitness Landscape Footprint: A Framework to Compare Neural Architecture Search Problems
- NAS-HPO-Bench-II: A Benchmark Dataset on Joint Optimization of Convolutional Neural Network Architecture and Training Hyperparameters
- Rethinking Neural Operations for Diverse Tasks
- Large Scale Neural Architecture Search with Polyharmonic Splines
- Yet Another Intermediate-Level Attack
- MS-DARTS: Mean-Shift Based Differentiable Architecture Search
- BaLeNAS: Differentiable Architecture Search via the Bayesian Learning Rule
- GPNAS: A Neural Network Architecture Search Framework Based on Graphical Predictor
- Connection Sensitivity Matters for Training-free DARTS: From Architecture-Level Scoring to Operation-Level Sensitivity Analysis
- An Analysis of Super-Net Heuristics in Weight-Sharing NAS