Learning to Optimize Neural Nets

Abstract

Learning to Optimize is a recently proposed framework for learning optimization algorithms using reinforcement learning. In this paper, we explore learning an optimization algorithm for training shallow neural nets. Such high-dimensional stochastic optimization problems present interesting challenges for existing reinforcement learning algorithms. We develop an extension that is suited to learning optimization algorithms in this setting and demonstrate that the learned optimization algorithm consistently outperforms other known optimization algorithms even on unseen tasks and is robust to changes in stochasticity of gradients and the neural net architecture. More specifically, we show that an optimization algorithm trained with the proposed method on the problem of training a neural net on MNIST generalizes to the problems of training neural nets on the Toronto Faces Dataset, CIFAR-10 and CIFAR-100.

10 pages, 15 figures

References in corpus (2)

• Practical Bayesian Optimization of Machine Learning Algorithms
• Gradient-based Hyperparameter Optimization through Reversible Learning

Cited by in corpus (27)

• Language Models are Few-Shot Learners
• Neural Optimizer Search with Reinforcement Learning
• Learning to Optimize: A Primer and A Benchmark
• Meta-Learning and Universality: Deep Representations and Gradient Descent can Approximate any Learning Algorithm
• Discriminative Adversarial Domain Generalization with Meta-learning based Cross-domain Validation
• Improving Generalization in Meta Reinforcement Learning using Learned Objectives
• Meta-learning Based Beamforming Design for MISO Downlink
• Truncated Back-propagation for Bilevel Optimization
• When MAML Can Adapt Fast and How to Assist When It Cannot
• Learning to Optimize in Model Predictive Control
• Using a thousand optimization tasks to learn hyperparameter search strategies
• Tasks, stability, architecture, and compute: Training more effective learned optimizers, and using them to train themselves
• On the Importance of Attention in Meta-Learning for Few-Shot Text Classification
• Meta-Learning Acquisition Functions for Transfer Learning in Bayesian Optimization
• Meta Learning Backpropagation And Improving It
• Learning a Prior over Intent via Meta-Inverse Reinforcement Learning
• MetaAlign: Coordinating Domain Alignment and Classification for Unsupervised Domain Adaptation
• Learning to Learn by Zeroth-Order Oracle
• Guarantees for Tuning the Step Size using a Learning-to-Learn Approach
• Learning to Optimise General TSP Instances
• Q-DeckRec: A Fast Deck Recommendation System for Collectible Card Games
• Neuro-Optimization: Learning Objective Functions Using Neural Networks
• Meta-Learning with Hessian-Free Approach in Deep Neural Nets Training
• Retrieve, Program, Repeat: Complex Knowledge Base Question Answering via Alternate Meta-learning
• Neural Conditional Gradients
• Efficient Reinforcement Learning Development with RLzoo
• MetalGAN: a Cluster-based Adaptive Training for Few-Shot Adversarial Colorization