Finding Competitive Network Architectures Within a Day Using UCT
arXiv:1712.07420 · doi:10.1109/DSAA.2018.00037
Abstract
The design of neural network architectures for a new data set is a laborious task which requires human deep learning expertise. In order to make deep learning available for a broader audience, automated methods for finding a neural network architecture are vital. Recently proposed methods can already achieve human expert level performances. However, these methods have run times of months or even years of GPU computing time, ignoring hardware constraints as faced by many researchers and companies. We propose the use of Monte Carlo planning in combination with two different UCT (upper confidence bound applied to trees) derivations to search for network architectures. We adapt the UCT algorithm to the needs of network architecture search by proposing two ways of sharing information between different branches of the search tree. In an empirical study we are able to demonstrate that this method is able to find competitive networks for MNIST, SVHN and CIFAR-10 in just a single GPU day. Extending the search time to five GPU days, we are able to outperform human architectures and our competitors which consider the same types of layers.
References in corpus (15)
- Adam: A Method for Stochastic Optimization
- Very Deep Convolutional Networks for Large-Scale Image Recognition
- Sequence to Sequence Learning with Neural Networks
- Fashion-MNIST: a Novel Image Dataset for Benchmarking Machine Learning Algorithms
- Practical Bayesian Optimization of Machine Learning Algorithms
- A Brief Survey of Deep Reinforcement Learning
- Neural Architecture Search with Reinforcement Learning
- Striving for Simplicity: The All Convolutional Net
- FitNets: Hints for Thin Deep Nets
- Large-Scale Evolution of Image Classifiers
- Scalable Bayesian Optimization Using Deep Neural Networks
- Designing Neural Network Architectures using Reinforcement Learning
- DeepArchitect: Automatically Designing and Training Deep Architectures
- Freeze-Thaw Bayesian Optimization
- Finding Competitive Network Architectures Within a Day Using UCT