NAS-Bench-101: Towards Reproducible Neural Architecture Search
arXiv:1902.09635
Abstract
Recent advances in neural architecture search (NAS) demand tremendous computational resources, which makes it difficult to reproduce experiments and imposes a barrier-to-entry to researchers without access to large-scale computation. We aim to ameliorate these problems by introducing NAS-Bench-101, the first public architecture dataset for NAS research. To build NAS-Bench-101, we carefully constructed a compact, yet expressive, search space, exploiting graph isomorphisms to identify 423k unique convolutional architectures. We trained and evaluated all of these architectures multiple times on CIFAR-10 and compiled the results into a large dataset of over 5 million trained models. This allows researchers to evaluate the quality of a diverse range of models in milliseconds by querying the pre-computed dataset. We demonstrate its utility by analyzing the dataset as a whole and by benchmarking a range of architecture optimization algorithms.
arXiv commentPublished in the Proceedings of the 36th International Conference on Machine Learning
Cited by in corpus (105)
- AutoML: A Survey of the State-of-the-Art
- A Survey on Evolutionary Neural Architecture Search
- NAS-FAS: Static-Dynamic Central Difference Network Search for Face Anti-Spoofing
- One-Shot Neural Architecture Search via Self-Evaluated Template Network
- NATS-Bench: Benchmarking NAS Algorithms for Architecture Topology and Size
- BRP-NAS: Prediction-based NAS using GCNs
- Best Practices for Scientific Research on Neural Architecture Search
- Automated Machine Learning on Graphs: A Survey
- Bridging the Gap between Sample-based and One-shot Neural Architecture Search with BONAS
- EEEA-Net: An Early Exit Evolutionary Neural Architecture Search
- A Comprehensive Survey on Hardware-Aware Neural Architecture Search
- AlphaX: eXploring Neural Architectures with Deep Neural Networks and Monte Carlo Tree Search
- Towards Green Automated Machine Learning: Status Quo and Future Directions
- EPE-NAS: Efficient Performance Estimation Without Training for Neural Architecture Search
- Semi-Supervised Neural Architecture Search
- ReLeQ: A Reinforcement Learning Approach for Deep Quantization of Neural Networks
- Neural Architecture Search without Training
- Re-Examining Linear Embeddings for High-Dimensional Bayesian Optimization
- Contrastive Self-supervised Neural Architecture Search
- Deeper Insights into Weight Sharing in Neural Architecture Search
- Does Unsupervised Architecture Representation Learning Help Neural Architecture Search?
- Tabular Benchmarks for Joint Architecture and Hyperparameter Optimization
- Sample-Efficient Neural Architecture Search by Learning Action Space
- Evaluating Efficient Performance Estimators of Neural Architectures
- DDPNAS: Efficient Neural Architecture Search via Dynamic Distribution Pruning
- Weight-Sharing Neural Architecture Search: A Battle to Shrink the Optimization Gap
- EvoPose2D: Pushing the Boundaries of 2D Human Pose Estimation using Accelerated Neuroevolution with Weight Transfer
- CARS: Continuous Evolution for Efficient Neural Architecture Search
- How to Train Your Super-Net: An Analysis of Training Heuristics in Weight-Sharing NAS
- Model-based Asynchronous Hyperparameter and Neural Architecture Search
- Differential Evolution for Neural Architecture Search
- Neural Predictor for Neural Architecture Search
- Neural Architecture Generator Optimization
- ANNETTE: Accurate Neural Network Execution Time Estimation with Stacked Models
- Evolutionary-Neural Hybrid Agents for Architecture Search
- Using a thousand optimization tasks to learn hyperparameter search strategies
- Towards NNGP-guided Neural Architecture Search
- Rethinking Bottleneck Structure for Efficient Mobile Network Design
- To Share or Not To Share: A Comprehensive Appraisal of Weight-Sharing
- Speedy Performance Estimation for Neural Architecture Search
- Stronger NAS with Weaker Predictors
- Bayesian Optimisation over Multiple Continuous and Categorical Inputs
- Zen-NAS: A Zero-Shot NAS for High-Performance Deep Image Recognition
- AttentiveNAS: Improving Neural Architecture Search via Attentive Sampling
- A Semi-Supervised Assessor of Neural Architectures
- NASGEM: Neural Architecture Search via Graph Embedding Method
- Optimal Transport Kernels for Sequential and Parallel Neural Architecture Search
- A Variational-Sequential Graph Autoencoder for Neural Architecture Performance Prediction
- Techniques for Automated Machine Learning
- Efficient Forward Architecture Search
- Standing on the Shoulders of Giants: Hardware and Neural Architecture Co-Search with Hot Start
- Neural Architecture Optimization with Graph VAE
- An Introduction to Neural Architecture Search for Convolutional Networks
- Computational catalyst discovery: Active classification through myopic multiscale sampling
- Few-shot Neural Architecture Search
- Auto-PyTorch Tabular: Multi-Fidelity MetaLearning for Efficient and Robust AutoDL
- CATE: Computation-aware Neural Architecture Encoding with Transformers
- Prune and Replace NAS
- Learning Versatile Neural Architectures by Propagating Network Codes
- Fitting the Search Space of Weight-sharing NAS with Graph Convolutional Networks
- Meta-Surrogate Benchmarking for Hyperparameter Optimization
- Accuracy Prediction with Non-neural Model for Neural Architecture Search
- Self-supervised Representation Learning for Evolutionary Neural Architecture Search
- Towards Learning of Filter-Level Heterogeneous Compression of Convolutional Neural Networks
- EH-DNAS: End-to-End Hardware-aware Differentiable Neural Architecture Search
- GENNAPE: Towards Generalized Neural Architecture Performance Estimators
- Contrastive Neural Architecture Search with Neural Architecture Comparators
- MUXConv: Information Multiplexing in Convolutional Neural Networks
- Searching for Stage-wise Neural Graphs In the Limit
- Bag of Tricks for Neural Architecture Search
- TransNAS-Bench-101: Improving Transferability and Generalizability of Cross-Task Neural Architecture Search
- Fine-Grained Stochastic Architecture Search
- Binarized Neural Architecture Search for Efficient Object Recognition
- A Study on Encodings for Neural Architecture Search
- Fitness Landscape Footprint: A Framework to Compare Neural Architecture Search Problems
- SCARLET-NAS: Bridging the Gap between Stability and Scalability in Weight-sharing Neural Architecture Search
- AIO-P: Expanding Neural Performance Predictors Beyond Image Classification
- Near-linear Time Gaussian Process Optimization with Adaptive Batching and Resparsification
- FEAR: A Simple Lightweight Method to Rank Architectures
- Binarized Neural Architecture Search
- Transfer NAS: Knowledge Transfer between Search Spaces with Transformer Agents
- Generic Neural Architecture Search via Regression
- NAS-HPO-Bench-II: A Benchmark Dataset on Joint Optimization of Convolutional Neural Network Architecture and Training Hyperparameters
- Exploring the Loss Landscape in Neural Architecture Search
- ProxyBO: Accelerating Neural Architecture Search via Bayesian Optimization with Zero-cost Proxies
- Hyperparameter optimization with REINFORCE and Transformers
- Rethinking Neural Operations for Diverse Tasks
- FixNorm: Dissecting Weight Decay for Training Deep Neural Networks
- Disentangled Neural Architecture Search
- BaLeNAS: Differentiable Architecture Search via the Bayesian Learning Rule
- Lessons from the Clustering Analysis of a Search Space: A Centroid-based Approach to Initializing NAS
- Efficient Sampling for Predictor-Based Neural Architecture Search
- Efficient Model Performance Estimation via Feature Histories
- Pretraining Neural Architecture Search Controllers with Locality-based Self-Supervised Learning
- Neural networks adapting to datasets: learning network size and topology
- Joint Learning of Neural Transfer and Architecture Adaptation for Image Recognition
- Enhanced Gradient for Differentiable Architecture Search
- Analyzing Neural Networks Based on Random Graphs
- An Analysis of Super-Net Heuristics in Weight-Sharing NAS
- Mutation is all you need
- Accelerating Neural Architecture Search via Proxy Data
- Training BatchNorm Only in Neural Architecture Search and Beyond
- LETI: Latency Estimation Tool and Investigation of Neural Networks inference on Mobile GPU
- Connection Sensitivity Matters for Training-free DARTS: From Architecture-Level Scoring to Operation-Level Sensitivity Analysis
- GPNAS: A Neural Network Architecture Search Framework Based on Graphical Predictor