Neural Architecture Search Over a Graph Search Space
arXiv:1812.10666
Abstract
Neural Architecture Search (NAS) enabled the discovery of state-of-the-art architectures in many domains. However, the success of NAS depends on the definition of the search space. Current search spaces are defined as a static sequence of decisions and a set of available actions for each decision. Each possible sequence of actions defines an architecture. We propose a more expressive class of search space: directed graphs. In our formalism, each decision is a vertex and each action is an edge. This allows us to model iterative and branching architecture design decisions. We demonstrate in simulation, and on image classification experiments, basic iterative and branching search structures, and show that the graph representation improves sample efficiency.
References in corpus (12)
- Neural Architecture Search with Reinforcement Learning
- DARTS: Differentiable Architecture Search
- Theoretical Models of Learning to Learn
- Efficient Neural Architecture Search via Parameter Sharing
- Deep Neuroevolution: Genetic Algorithms Are a Competitive Alternative for Training Deep Neural Networks for Reinforcement Learning
- SMASH: One-Shot Model Architecture Search through HyperNetworks
- Neural Optimizer Search with Reinforcement Learning
- Evaluating the Search Phase of Neural Architecture Search
- Progressive Neural Architecture Search
- DeepArchitect: Automatically Designing and Training Deep Architectures
- Raiders of the Lost Architecture: Kernels for Bayesian Optimization in Conditional Parameter Spaces
- Neural Program Synthesis with Priority Queue Training