MANAS: Multi-Agent Neural Architecture Search
arXiv:1909.01051
Abstract
The Neural Architecture Search (NAS) problem is typically formulated as a graph search problem where the goal is to learn the optimal operations over edges in order to maximise a graph-level global objective. Due to the large architecture parameter space, efficiency is a key bottleneck preventing NAS from its practical use. In this paper, we address the issue by framing NAS as a multi-agent problem where agents control a subset of the network and coordinate to reach optimal architectures. We provide two distinct lightweight implementations, with reduced memory requirements (1/8th of state-of-the-art), and performances above those of much more computationally expensive methods. Theoretically, we demonstrate vanishing regrets of the form O(sqrt(T)), with T being the total number of rounds. Finally, aware that random search is an, often ignored, effective baseline we perform additional experiments on 3 alternative datasets and 2 network configurations, and achieve favourable results in comparison.
References in corpus (6)
- Neural Architecture Search with Reinforcement Learning
- DARTS: Differentiable Architecture Search
- AutoAugment: Learning Augmentation Policies from Data
- QMIX: Monotonic Value Function Factorisation for Deep Multi-Agent Reinforcement Learning
- Random Search and Reproducibility for Neural Architecture Search
- NAS evaluation is frustratingly hard
Cited by in corpus (10)
- NAS evaluation is frustratingly hard
- BNAS:An Efficient Neural Architecture Search Approach Using Broad Scalable Architecture
- Weight-Sharing Neural Architecture Search: A Battle to Shrink the Optimization Gap
- Neural Architecture Generator Optimization
- Bayesian Optimisation over Multiple Continuous and Categorical Inputs
- Batch Group Normalization
- DHA: End-to-End Joint Optimization of Data Augmentation Policy, Hyper-parameter and Architecture
- Direct Federated Neural Architecture Search
- Revisiting Neural Architecture Search
- AUTOKD: Automatic Knowledge Distillation Into A Student Architecture Family