Designing Neural Network Architectures using Reinforcement Learning

Abstract

At present, designing convolutional neural network (CNN) architectures requires both human expertise and labor. New architectures are handcrafted by careful experimentation or modified from a handful of existing networks. We introduce MetaQNN, a meta-modeling algorithm based on reinforcement learning to automatically generate high-performing CNN architectures for a given learning task. The learning agent is trained to sequentially choose CNN layers using $Q$-learning with an $\epsilon$-greedy exploration strategy and experience replay. The agent explores a large but finite space of possible architectures and iteratively discovers designs with improved performance on the learning task. On image classification benchmarks, the agent-designed networks (consisting of only standard convolution, pooling, and fully-connected layers) beat existing networks designed with the same layer types and are competitive against the state-of-the-art methods that use more complex layer types. We also outperform existing meta-modeling approaches for network design on image classification tasks.

Cited by in corpus (11)

• A Survey on Neural Architecture Search
• Progressive Differentiable Architecture Search: Bridging the Depth Gap between Search and Evaluation
• Routing Networks and the Challenges of Modular and Compositional Computation
• Routing Networks: Adaptive Selection of Non-linear Functions for Multi-Task Learning
• Tabular Benchmarks for Joint Architecture and Hyperparameter Optimization
• HyperNOMAD: Hyperparameter optimization of deep neural networks using mesh adaptive direct search
• StyleNAS: An Empirical Study of Neural Architecture Search to Uncover Surprisingly Fast End-to-End Universal Style Transfer Networks
• Transfer NAS: Knowledge Transfer between Search Spaces with Transformer Agents
• Adaptive Bayesian Linear Regression for Automated Machine Learning
• WeNet: Weighted Networks for Recurrent Network Architecture Search
• Automated Search for Configurations of Deep Neural Network Architectures