Designing Neural Network Architectures using Reinforcement Learning

Abstract

At present, designing convolutional neural network (CNN) architectures requires both human expertise and labor. New architectures are handcrafted by careful experimentation or modified from a handful of existing networks. We introduce MetaQNN, a meta-modeling algorithm based on reinforcement learning to automatically generate high-performing CNN architectures for a given learning task. The learning agent is trained to sequentially choose CNN layers using $Q$-learning with an $\epsilon$-greedy exploration strategy and experience replay. The agent explores a large but finite space of possible architectures and iteratively discovers designs with improved performance on the learning task. On image classification benchmarks, the agent-designed networks (consisting of only standard convolution, pooling, and fully-connected layers) beat existing networks designed with the same layer types and are competitive against the state-of-the-art methods that use more complex layer types. We also outperform existing meta-modeling approaches for network design on image classification tasks.

Cited by in corpus (142)

• Neural Architecture Search: A Survey
• AutoML: A Survey of the State-of-the-Art
• Automatically designing CNN architectures using genetic algorithm for image classification
• PC-DARTS: Partial Channel Connections for Memory-Efficient Architecture Search
• Understanding and Robustifying Differentiable Architecture Search
• One-Shot Neural Architecture Search via Self-Evaluated Template Network
• A Survey on Neural Architecture Search
• A Particle Swarm Optimization-based Flexible Convolutional Auto-Encoder for Image Classification
• Differentiable Quantum Architecture Search
• Path-Level Network Transformation for Efficient Architecture Search
• Auto-DeepLab: Hierarchical Neural Architecture Search for Semantic Image Segmentation
• Progressive Differentiable Architecture Search: Bridging the Depth Gap between Search and Evaluation
• AutoML-Zero: Evolving Machine Learning Algorithms From Scratch
• Path-Restore: Learning Network Path Selection for Image Restoration
• Graph HyperNetworks for Neural Architecture Search
• BRP-NAS: Prediction-based NAS using GCNs
• Neural Predictor based Quantum Architecture Search
• Meta-Learning Update Rules for Unsupervised Representation Learning
• Evolving Deep Convolutional Neural Networks for Image Classification
• NSGA-Net: Neural Architecture Search using Multi-Objective Genetic Algorithm
• Efficient Multi-objective Neural Architecture Search via Lamarckian Evolution
• Evolving Normalization-Activation Layers
• Predicting atmospheric optical properties for radiative transfer computations using neural networks
• Normalization Techniques in Training DNNs: Methodology, Analysis and Application
• Efficient Neural Architecture Transformation Searchin Channel-Level for Object Detection
• EPE-NAS: Efficient Performance Estimation Without Training for Neural Architecture Search
• Scalable Deep Learning on Distributed Infrastructures: Challenges, Techniques and Tools
• Routing Networks and the Challenges of Modular and Compositional Computation
• Multi-Objective Reinforced Evolution in Mobile Neural Architecture Search
• A Comprehensive Overview and Survey of Recent Advances in Meta-Learning
• Routing Networks: Adaptive Selection of Non-linear Functions for Multi-Task Learning
• Deeper Insights into Weight Sharing in Neural Architecture Search
• Does Unsupervised Architecture Representation Learning Help Neural Architecture Search?
• Tabular Benchmarks for Joint Architecture and Hyperparameter Optimization
• Finding Competitive Network Architectures Within a Day Using UCT
• Reinforced Evolutionary Neural Architecture Search
• Towards Optimal Structured CNN Pruning via Generative Adversarial Learning
• Weight-Sharing Neural Architecture Search: A Battle to Shrink the Optimization Gap
• EvoPose2D: Pushing the Boundaries of 2D Human Pose Estimation using Accelerated Neuroevolution with Weight Transfer
• Search to aggregate neighborhood for graph neural network
• Sub-Architecture Ensemble Pruning in Neural Architecture Search
• Latency-Aware Differentiable Neural Architecture Search
• AutoHAS: Efficient Hyperparameter and Architecture Search
• Automatically Evolving CNN Architectures Based on Blocks
• DCNAS: Densely Connected Neural Architecture Search for Semantic Image Segmentation
• Lightweight Residual Densely Connected Convolutional Neural Network
• CurveLane-NAS: Unifying Lane-Sensitive Architecture Search and Adaptive Point Blending
• When NAS Meets Robustness: In Search of Robust Architectures against Adversarial Attacks
• Differentiable Neural Input Search for Recommender Systems
• Discovering General-Purpose Active Learning Strategies
• C2FNAS: Coarse-to-Fine Neural Architecture Search for 3D Medical Image Segmentation
• Efficient Visual Recognition with Deep Neural Networks: A Survey on Recent Advances and New Directions
• Transformer-based Context Condensation for Boosting Feature Pyramids in Object Detection
• Automated Concatenation of Embeddings for Structured Prediction
• Multi-Objective Evolutionary Design of Deep Convolutional Neural Networks for Image Classification
• Parallel Architecture and Hyperparameter Search via Successive Halving and Classification
• PILAE: A Non-gradient Descent Learning Scheme for Deep Feedforward Neural Networks
• Interstellar: Searching Recurrent Architecture for Knowledge Graph Embedding
• Learning to reinforcement learn for Neural Architecture Search
• SalNAS: Efficient Saliency-prediction Neural Architecture Search with self-knowledge distillation
• Zen-NAS: A Zero-Shot NAS for High-Performance Deep Image Recognition
• Automated Design Space Exploration for optimised Deployment of DNN on Arm Cortex-A CPUs
• M${}^3$RL: Mind-aware Multi-agent Management Reinforcement Learning
• Can weight sharing outperform random architecture search? An investigation with TuNAS
• Efficient Neural Architecture Search via Proximal Iterations
• Optimal Transport Kernels for Sequential and Parallel Neural Architecture Search
• HyperNOMAD: Hyperparameter optimization of deep neural networks using mesh adaptive direct search
• ReinBo: Machine Learning pipeline search and configuration with Bayesian Optimization embedded Reinforcement Learning
• Searching to Exploit Memorization Effect in Learning from Corrupted Labels
• Few-shot Neural Architecture Search
• AutoCaption: Image Captioning with Neural Architecture Search
• Improving One-shot NAS by Suppressing the Posterior Fading
• Neural Architecture Search for Deep Image Prior
• TF-NAS: Rethinking Three Search Freedoms of Latency-Constrained Differentiable Neural Architecture Search
• CATE: Computation-aware Neural Architecture Encoding with Transformers
• AttentionNAS: Spatiotemporal Attention Cell Search for Video Classification
• Inductive Transfer for Neural Architecture Optimization
• Multi-objective Neural Architecture Search via Non-stationary Policy Gradient
• HM-NAS: Efficient Neural Architecture Search via Hierarchical Masking
• On the Privacy Risks of Cell-Based NAS Architectures
• Rapid Neural Architecture Search by Learning to Generate Graphs from Datasets
• NeuNetS: An Automated Synthesis Engine for Neural Network Design
• Neural Architecture Search by Estimation of Network Structure Distributions
• Contrastive Neural Architecture Search with Neural Architecture Comparators
• EH-DNAS: End-to-End Hardware-aware Differentiable Neural Architecture Search
• Auto-MVCNN: Neural Architecture Search for Multi-view 3D Shape Recognition
• Integrating Large Circular Kernels into CNNs through Neural Architecture Search
• NPAS: A Compiler-aware Framework of Unified Network Pruning and Architecture Search for Beyond Real-Time Mobile Acceleration
• Exploiting Channel Similarity for Accelerating Deep Convolutional Neural Networks
• A Novel Training Protocol for Performance Predictors of Evolutionary Neural Architecture Search Algorithms
• Auto-Classifier: A Robust Defect Detector Based on an AutoML Head
• Searching for Fast Model Families on Datacenter Accelerators
• Generalization of Model-Agnostic Meta-Learning Algorithms: Recurring and Unseen Tasks
• NAS-DIP: Learning Deep Image Prior with Neural Architecture Search
• Epigenetic evolution of deep convolutional models
• Slimmable Compressive Autoencoders for Practical Neural Image Compression
• PONAS: Progressive One-shot Neural Architecture Search for Very Efficient Deployment
• Efficient Synthesis of Compact Deep Neural Networks
• Multi-Objective Neural Architecture Search Based on Diverse Structures and Adaptive Recommendation
• Disturbance-immune Weight Sharing for Neural Architecture Search
• Search For Deep Graph Neural Networks
• Phoebe: Reuse-Aware Online Caching with Reinforcement Learning for Emerging Storage Models
• Neighborhood-Aware Neural Architecture Search
• Reinforcement Learning for Flexibility Design Problems
• Deep-n-Cheap: An Automated Search Framework for Low Complexity Deep Learning
• NAS-HPO-Bench-II: A Benchmark Dataset on Joint Optimization of Convolutional Neural Network Architecture and Training Hyperparameters
• Training convolutional neural networks with cheap convolutions and online distillation
• Efficient Neural Interaction Function Search for Collaborative Filtering
• WeNet: Weighted Networks for Recurrent Network Architecture Search
• Adaptive Bayesian Linear Regression for Automated Machine Learning
• StyleNAS: An Empirical Study of Neural Architecture Search to Uncover Surprisingly Fast End-to-End Universal Style Transfer Networks
• Transfer NAS: Knowledge Transfer between Search Spaces with Transformer Agents
• NASIB: Neural Architecture Search withIn Budget
• HMCNAS: Neural Architecture Search using Hidden Markov Chains and Bayesian Optimization
• Automated Search for Configurations of Deep Neural Network Architectures
• Deep Reinforced Attention Learning for Quality-Aware Visual Recognition
• Towards Accurate and Compact Architectures via Neural Architecture Transformer
• Learning Architectures from an Extended Search Space for Language Modeling
• What and Where: Learn to Plug Adapters via NAS for Multi-Domain Learning
• An Efficient Quantitative Approach for Optimizing Convolutional Neural Networks
• ABCP: Automatic Block-wise and Channel-wise Network Pruning via Joint Search
• Searching for A Robust Neural Architecture in Four GPU Hours
• Dynamical System Inspired Adaptive Time Stepping Controller for Residual Network Families
• Deep Learning on Attributed Graphs: A Journey from Graphs to Their Embeddings and Back
• iDARTS: Improving DARTS by Node Normalization and Decorrelation Discretization
• Fuzzy Logic Interpretation of Quadratic Networks
• Fully Dynamic Inference with Deep Neural Networks
• Structured Sparsification with Joint Optimization of Group Convolution and Channel Shuffle
• Optimization with learning-informed differential equation constraints and its applications
• Autoequivariant Network Search via Group Decomposition
• Learning from Mistakes based on Class Weighting with Application to Neural Architecture Search
• GAIA: A Transfer Learning System of Object Detection that Fits Your Needs
• Joint Learning of Neural Transfer and Architecture Adaptation for Image Recognition
• Neural networks adapting to datasets: learning network size and topology
• GIID-Net: Generalizable Image Inpainting Detection via Neural Architecture Search and Attention
• Analyzing Neural Networks Based on Random Graphs
• Searching by Generating: Flexible and Efficient One-Shot NAS with Architecture Generator
• Layer Flexible Adaptive Computational Time
• GPNAS: A Neural Network Architecture Search Framework Based on Graphical Predictor
• On the Security Risks of AutoML
• Neural Architecture Search based on Cartesian Genetic Programming Coding Method
• Adaptive Remote Sensing Image Attribute Learning for Active Object Detection