Guarantees for Tuning the Step Size using a Learning-to-Learn Approach
arXiv:2006.16495
Abstract
Choosing the right parameters for optimization algorithms is often the key to their success in practice. Solving this problem using a learning-to-learn approach -- using meta-gradient descent on a meta-objective based on the trajectory that the optimizer generates -- was recently shown to be effective. However, the meta-optimization problem is difficult. In particular, the meta-gradient can often explode/vanish, and the learned optimizer may not have good generalization performance if the meta-objective is not chosen carefully. In this paper we give meta-optimization guarantees for the learning-to-learn approach on a simple problem of tuning the step size for quadratic loss. Our results show that the naïve objective suffers from meta-gradient explosion/vanishing problem. Although there is a way to design the meta-objective so that the meta-gradient remains polynomially bounded, computing the meta-gradient directly using backpropagation leads to numerical issues. We also characterize when it is necessary to compute the meta-objective on a separate validation set to ensure the generalization performance of the learned optimizer. Finally, we verify our results empirically and show that a similar phenomenon appears even for more complicated learned optimizers parametrized by neural networks.
ICML 2021. Added proof sketch
References in corpus (8)
- Meta-SGD: Learning to Learn Quickly for Few-Shot Learning
- RL: Fast Reinforcement Learning via Slow Reinforcement Learning
- Stochastic Gradient Descent for Non-smooth Optimization: Convergence Results and Optimal Averaging Schemes
- Learning to reinforcement learn
- Neural Optimizer Search with Reinforcement Learning
- Tight Analyses for Non-Smooth Stochastic Gradient Descent
- How Important is the Train-Validation Split in Meta-Learning?
- Learning to Prune: Speeding up Repeated Computations