Evolutionary-Neural Hybrid Agents for Architecture Search
arXiv:1811.09828
Abstract
Neural Architecture Search has shown potential to automate the design of neural networks. Deep Reinforcement Learning based agents can learn complex architectural patterns, as well as explore a vast and compositional search space. On the other hand, evolutionary algorithms offer higher sample efficiency, which is critical for such a resource intensive application. In order to capture the best of both worlds, we propose a class of Evolutionary-Neural hybrid agents (Evo-NAS). We show that the Evo-NAS agent outperforms both neural and evolutionary agents when applied to architecture search for a suite of text and image classification benchmarks. On a high-complexity architecture search space for image classification, the Evo-NAS agent surpasses the accuracy achieved by commonly used agents with only 1/3 of the search cost.
References in corpus (20)
- Adam: A Method for Stochastic Optimization
- MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applications
- Neural Architecture Search with Reinforcement Learning
- DARTS: Differentiable Architecture Search
- Universal Sentence Encoder
- ShuffleNet: An Extremely Efficient Convolutional Neural Network for Mobile Devices
- AutoAugment: Learning Augmentation Policies from Data
- Efficient Neural Architecture Search via Parameter Sharing
- Deep Neuroevolution: Genetic Algorithms Are a Competitive Alternative for Training Deep Neural Networks for Reinforcement Learning
- SNAS: Stochastic Neural Architecture Search
- NAS-Bench-101: Towards Reproducible Neural Architecture Search
- Random Search and Reproducibility for Neural Architecture Search
- Neural Optimizer Search with Reinforcement Learning
- The Evolved Transformer
- Simple And Efficient Architecture Search for Convolutional Neural Networks
- Progressive Neural Architecture Search
- DeepArchitect: Automatically Designing and Training Deep Architectures
- AdaNet: Adaptive Structural Learning of Artificial Neural Networks
- Reinforced Evolutionary Neural Architecture Search
- Neural Program Synthesis with Priority Queue Training
Cited by in corpus (10)
- AutoML: A Survey of the State-of-the-Art
- A Survey on Evolutionary Neural Architecture Search
- Hypernetwork functional image representation
- Weight-Sharing Neural Architecture Search: A Battle to Shrink the Optimization Gap
- How Powerful are Performance Predictors in Neural Architecture Search?
- Flexible Multi-task Networks by Learning Parameter Allocation
- A Study on Encodings for Neural Architecture Search
- Exploring the Loss Landscape in Neural Architecture Search
- Training BatchNorm Only in Neural Architecture Search and Beyond
- Joint Learning of Neural Transfer and Architecture Adaptation for Image Recognition