Neural Optimizer Search with Reinforcement Learning

Abstract

We present an approach to automate the process of discovering optimization methods, with a focus on deep learning architectures. We train a Recurrent Neural Network controller to generate a string in a domain specific language that describes a mathematical update equation based on a list of primitive functions, such as the gradient, running average of the gradient, etc. The controller is trained with Reinforcement Learning to maximize the performance of a model after a few epochs. On CIFAR-10, our method discovers several update rules that are better than many commonly used optimizers, such as Adam, RMSProp, or SGD with and without Momentum on a ConvNet model. We introduce two new optimizers, named PowerSign and AddSign, which we show transfer well and improve training on a variety of different tasks and architectures, including ImageNet classification and Google's neural machine translation system.

ICML 2017 Conference paper

References in corpus (8)

• ADADELTA: An Adaptive Learning Rate Method
• Neural Architecture Search with Reinforcement Learning
• Wide Residual Networks
• SGDR: Stochastic Gradient Descent with Warm Restarts
• No More Pesky Learning Rates
• Adding Gradient Noise Improves Learning for Very Deep Networks
• Learning to Optimize
• Learning to Optimize Neural Nets

Cited by in corpus (58)

• Searching for Activation Functions
• AutoAugment: Learning Augmentation Policies from Data
• Efficient Neural Architecture Search via Parameter Sharing
• Deep Reinforcement Learning: An Overview
• Hyper-Parameter Optimization: A Review of Algorithms and Applications
• One-Shot Neural Architecture Search via Self-Evaluated Template Network
• Reinforced Continual Learning
• AutoML-Zero: Evolving Machine Learning Algorithms From Scratch
• Review: Deep Learning in Electron Microscopy
• Coin.AI: A Proof-of-Useful-Work Scheme for Blockchain-based Distributed Deep Learning
• Learning to Optimize: A Primer and A Benchmark
• Meta-Learning Update Rules for Unsupervised Representation Learning
• Deep Bayesian Inversion
• GraphNAS: Graph Neural Architecture Search with Reinforcement Learning
• Deep symbolic regression: Recovering mathematical expressions from data via risk-seeking policy gradients
• Descending through a Crowded Valley - Benchmarking Deep Learning Optimizers
• Towards Explainable Personalized Recommendations by Learning from Users' Photos
• Learning to Handle Parameter Perturbations in Combinatorial Optimization: an Application to Facility Location
• Edge-labeling Graph Neural Network for Few-shot Learning
• AutoLoss: Learning Discrete Schedules for Alternate Optimization
• Quantitative Phase Imaging and Artificial Intelligence: A Review
• Deep Reinforcement Learning for Unmanned Aerial Vehicle-Assisted Vehicular Networks
• The model of an anomaly detector for HiLumi LHC magnets based on Recurrent Neural Networks and adaptive quantization
• Evolving Reinforcement Learning Algorithms
• HARK Side of Deep Learning -- From Grad Student Descent to Automated Machine Learning
• Auto-Panoptic: Cooperative Multi-Component Architecture Search for Panoptic Segmentation
• How to Train Your MAML to Excel in Few-Shot Classification
• Meta-learning curiosity algorithms
• Evolutionary-Neural Hybrid Agents for Architecture Search
• Learning to Optimize in Model Predictive Control
• Tasks, stability, architecture, and compute: Training more effective learned optimizers, and using them to train themselves
• Using a thousand optimization tasks to learn hyperparameter search strategies
• Using learned optimizers to make models robust to input noise
• Learning To Simulate
• Actor-Critic Reinforcement Learning for Control with Stability Guarantee
• Non-asymptotic Convergence of Adam-type Reinforcement Learning Algorithms under Markovian Sampling
• A Novel Framework for Neural Architecture Search in the Hill Climbing Domain
• MiLeNAS: Efficient Neural Architecture Search via Mixed-Level Reformulation
• Compiler-Level Matrix Multiplication Optimization for Deep Learning
• Training Stronger Baselines for Learning to Optimize
• Transfer Learning to Learn with Multitask Neural Model Search
• Guarantees for Tuning the Step Size using a Learning-to-Learn Approach
• ARDA: Automatic Relational Data Augmentation for Machine Learning
• Neural Architecture Search Over a Graph Search Space
• Neural Architecture Refinement: A Practical Way for Avoiding Overfitting in NAS
• Hyper-parameter Tuning under a Budget Constraint
• Reverse engineering learned optimizers reveals known and novel mechanisms
• Training Learned Optimizers with Randomly Initialized Learned Optimizers
• On the Orthogonality of Knowledge Distillation with Other Techniques: From an Ensemble Perspective
• Acceleration via Fractal Learning Rate Schedules
• CompNet: Neural networks growing via the compact network morphism
• Single-level Optimization For Differential Architecture Search
• Gradient-only line searches to automatically determine learning rates for a variety of stochastic training algorithms
• AE-Netv2: Optimization of Image Fusion Efficiency and Network Architecture
• Layout Design for Intelligent Warehouse by Evolution with Fitness Approximation
• T-AutoML: Automated Machine Learning for Lesion Segmentation using Transformers in 3D Medical Imaging
• Single-DARTS: Towards Stable Architecture Search
• Extreme Tensoring for Low-Memory Preconditioning