Learning to Learn without Gradient Descent by Gradient Descent

Abstract

We learn recurrent neural network optimizers trained on simple synthetic functions by gradient descent. We show that these learned optimizers exhibit a remarkable degree of transfer in that they can be used to efficiently optimize a broad range of derivative-free black-box functions, including Gaussian process bandits, simple control objectives, global optimization benchmarks and hyper-parameter tuning tasks. Up to the training horizon, the learned optimizers learn to trade-off exploration and exploitation, and compare favourably with heavily engineered Bayesian optimization packages for hyper-parameter tuning.

Accepted by ICML 2017. Previous version "Learning to Learn for Global Optimization of Black Box Functions" was published in the Deep Reinforcement Learning Workshop, NIPS 2016

Cited by in corpus (57)

• Causal Discovery with Reinforcement Learning
• Automated Machine Learning in Practice: State of the Art and Recent Results
• Sample Efficient Adaptive Text-to-Speech
• Automated Reinforcement Learning (AutoRL): A Survey and Open Problems
• Learning to Optimize: A Primer and A Benchmark
• Meta-Learning Update Rules for Unsupervised Representation Learning
• Meta-Learning with Warped Gradient Descent
• Learning Gradient Descent: Better Generalization and Longer Horizons
• Efficient Video Object Segmentation via Network Modulation
• LFPT5: A Unified Framework for Lifelong Few-shot Language Learning Based on Prompt Tuning of T5
• AutoLoss: Learning Discrete Schedules for Alternate Optimization
• Learning Surrogate Losses
• Learning to Warm-Start Bayesian Hyperparameter Optimization
• LS-Net: Learning to Solve Nonlinear Least Squares for Monocular Stereo
• Discriminative Optimization: Theory and Applications to Computer Vision Problems
• Unsupervised Learning of Neural Networks to Explain Neural Networks
• RAGO: Recurrent Graph Optimizer For Multiple Rotation Averaging
• Learning to Optimize in Model Predictive Control
• Learning to Optimize in Swarms
• Meta-Learning surrogate models for sequential decision making
• Meta-Learning Acquisition Functions for Transfer Learning in Bayesian Optimization
• Automated Learning Rate Scheduler for Large-batch Training
• L${}^2$-GCN: Layer-Wise and Learned Efficient Training of Graph Convolutional Networks
• The Differentiable Cross-Entropy Method
• Towards Assessing the Impact of Bayesian Optimization's Own Hyperparameters
• Meta-Surrogate Benchmarking for Hyperparameter Optimization
• Learning to Learn by Zeroth-Order Oracle
• Network Transplanting
• Training Stronger Baselines for Learning to Optimize
• Learning to Learn in a Semi-Supervised Fashion
• Rectified Meta-Learning from Noisy Labels for Robust Image-based Plant Disease Diagnosis
• Meta Learning Black-Box Population-Based Optimizers
• Q-DeckRec: A Fast Deck Recommendation System for Collectible Card Games
• Expert-Calibrated Learning for Online Optimization with Switching Costs
• Graceful Degradation and Related Fields
• Towards White-box Benchmarks for Algorithm Control
• Learning to be Global Optimizer
• Population-Based Evolution Optimizes a Meta-Learning Objective
• Learning Neural Activations
• Model-Agnostic Meta-Learning using Runge-Kutta Methods
• Reinforcement Learning To Adapt Speech Enhancement to Instantaneous Input Signal Quality
• Meta-Learning with Hessian-Free Approach in Deep Neural Nets Training
• Meta-Learning the Search Distribution of Black-Box Random Search Based Adversarial Attacks
• Collaborative Sampling in Generative Adversarial Networks
• Model-Agnostic Meta-Attack: Towards Reliable Evaluation of Adversarial Robustness
• Learning to Optimize with Dynamic Mode Decomposition
• On Hyper-parameter Tuning for Stochastic Optimization Algorithms
• Recurrent machines for likelihood-free inference
• Augmenting Supervised Learning by Meta-learning Unsupervised Local Rules
• Neural Conditional Gradients
• One-Shot Generation of Near-Optimal Topology through Theory-Driven Machine Learning
• ModelPred: A Framework for Predicting Trained Model from Training Data
• Learning to Search for MIMO Detection
• FISAR: Forward Invariant Safe Reinforcement Learning with a Deep Neural Network-Based Optimize
• Task Attended Meta-Learning for Few-Shot Learning
• Finding Significant Features for Few-Shot Learning using Dimensionality Reduction
• MTL2L: A Context Aware Neural Optimiser