Learning to Optimize

Abstract

Algorithm design is a laborious process and often requires many iterations of ideation and validation. In this paper, we explore automating algorithm design and present a method to learn an optimization algorithm, which we believe to be the first method that can automatically discover a better algorithm. We approach this problem from a reinforcement learning perspective and represent any particular optimization algorithm as a policy. We learn an optimization algorithm using guided policy search and demonstrate that the resulting algorithm outperforms existing hand-engineered algorithms in terms of convergence speed and/or the final objective value.

9 pages, 3 figures

References in corpus (11)

• Practical Bayesian Optimization of Machine Learning Algorithms
• A Tutorial on Bayesian Optimization of Expensive Cost Functions, with Application to Active User Modeling and Hierarchical Reinforcement Learning
• End-to-End Training of Deep Visuomotor Policies
• Theoretical Models of Learning to Learn
• Gradient-based Hyperparameter Optimization through Reversible Learning
• Neural Programmer-Interpreters
• Neural GPUs Learn Algorithms
• Reinforcement Learning Neural Turing Machines - Revised
• Learning Contact-Rich Manipulation Skills with Guided Policy Search
• Learning Simple Algorithms from Examples
• Lie Access Neural Turing Machine

Cited by in corpus (80)

• Neural Architecture Search with Reinforcement Learning
• Learning to Optimize: Training Deep Neural Networks for Wireless Resource Management
• Meta-SGD: Learning to Learn Quickly for Few-Shot Learning
• Learning to reinforcement learn
• Neural Optimizer Search with Reinforcement Learning
• Continuous Adaptation via Meta-Learning in Nonstationary and Competitive Environments
• Bayesian Recurrent Neural Networks
• Evolved Policy Gradients
• Review: Deep Learning in Electron Microscopy
• Automated Machine Learning in Practice: State of the Art and Recent Results
• Learning to Adapt in Dynamic, Real-World Environments Through Meta-Reinforcement Learning
• Meta-Learning and Universality: Deep Representations and Gradient Descent can Approximate any Learning Algorithm
• How to train your MAML
• Learning to Draw Samples: With Application to Amortized MLE for Generative Adversarial Learning
• Learning to Optimize Neural Nets
• Meta-Learning with Warped Gradient Descent
• Improving Generalization in Meta Reinforcement Learning using Learned Objectives
• Small Sample Learning in Big Data Era
• Learning What Data to Learn
• AI-GAs: AI-generating algorithms, an alternate paradigm for producing general artificial intelligence
• A Helmholtz equation solver using unsupervised learning: Application to transcranial ultrasound
• Meta-learning of Sequential Strategies
• AutoLoss: Learning Discrete Schedules for Alternate Optimization
• Weight-Sharing Neural Architecture Search: A Battle to Shrink the Optimization Gap
• Meta-learning Based Beamforming Design for MISO Downlink
• The model of an anomaly detector for HiLumi LHC magnets based on Recurrent Neural Networks and adaptive quantization
• Accelerating Quadratic Optimization with Reinforcement Learning
• Unsupervised Learning of Neural Networks to Explain Neural Networks
• Learning to superoptimize programs
• A Learning-boosted Quasi-Newton Method for AC Optimal Power Flow
• When MAML Can Adapt Fast and How to Assist When It Cannot
• Learning to Optimize in Swarms
• Measuring Outcomes in Healthcare Economics using Artificial Intelligence: with Application to Resource Management
• Reinforcement-Learning-Based Variational Quantum Circuits Optimization for Combinatorial Problems
• Learning to learn via Self-Critique
• Meta-Learning Acquisition Functions for Transfer Learning in Bayesian Optimization
• Meta-Learning for Contextual Bandit Exploration
• Generating Neural Networks with Neural Networks
• Learning Deep Gradient Descent Optimization for Image Deconvolution
• Multi-Objective Meta Learning
• Meta Learning Backpropagation And Improving It
• L${}^2$-GCN: Layer-Wise and Learned Efficient Training of Graph Convolutional Networks
• Structured Prediction for Conditional Meta-Learning
• The Differentiable Cross-Entropy Method
• Learning State-Dependent Losses for Inverse Dynamics Learning
• Efficient Decoupled Neural Architecture Search by Structure and Operation Sampling
• Calibration of Shared Equilibria in General Sum Partially Observable Markov Games
• Learning to Learn by Zeroth-Order Oracle
• Training Stronger Baselines for Learning to Optimize
• Guarantees for Tuning the Step Size using a Learning-to-Learn Approach
• Learning to Discretize: Solving 1D Scalar Conservation Laws via Deep Reinforcement Learning
• Graceful Degradation and Related Fields
• Stochastic Anderson Mixing for Nonconvex Stochastic Optimization
• Safeguarded Learned Convex Optimization
• A Generative Model for Sampling High-Performance and Diverse Weights for Neural Networks
• GLAD: Learning Sparse Graph Recovery
• Meta Adversarial Perturbations
• Deep Unfolding of a Proximal Interior Point Method for Image Restoration
• On the Convergence of Learning-based Iterative Methods for Nonconvex Inverse Problems
• Meta-Learning with Hessian-Free Approach in Deep Neural Nets Training
• Gradient Importance Learning for Incomplete Observations
• Learning the Step-size Policy for the Limited-Memory Broyden-Fletcher-Goldfarb-Shanno Algorithm
• Reverse engineering learned optimizers reveals known and novel mechanisms
• Iterative Teaching by Label Synthesis
• Learning Enhanced Optimisation for Routing Problems
• Fuzzy Logic Interpretation of Quadratic Networks
• Neural Fixed-Point Acceleration for Convex Optimization
• Meta-Learning Guarantees for Online Receding Horizon Learning Control
• A Meta-Learning Control Algorithm with Provable Finite-Time Guarantees
• Learning to Search for MIMO Detection
• Finding Significant Features for Few-Shot Learning using Dimensionality Reduction
• Few-Shot Meta-Denoising
• Meta-Learning with Adjoint Methods
• Self-Learning Tuning for Post-Silicon Validation
• ModelPred: A Framework for Predicting Trained Model from Training Data
• Meta-Learning a Dynamical Language Model
• Learning to Initialize Gradient Descent Using Gradient Descent
• Learning to Optimize Under Constraints with Unsupervised Deep Neural Networks
• Generalized Reinforcement Meta Learning for Few-Shot Optimization
• Consistency Guided Scene Flow Estimation