Generalized Inner Loop Meta-Learning

Abstract

Many (but not all) approaches self-qualifying as "meta-learning" in deep learning and reinforcement learning fit a common pattern of approximating the solution to a nested optimization problem. In this paper, we give a formalization of this shared pattern, which we call GIMLI, prove its general requirements, and derive a general-purpose algorithm for implementing similar approaches. Based on this analysis and algorithm, we describe a library of our design, higher, which we share with the community to assist and enable future research into these kinds of meta-learning approaches. We end the paper by showcasing the practical applications of this framework and library through illustrative experiments and ablation studies which they facilitate.

17 pages, 3 figures, 1 algorithm

References in corpus (3)

• Very Deep Convolutional Networks for Large-Scale Image Recognition
• Practical Bayesian Optimization of Machine Learning Algorithms
• RL${}^2$: Fast Reinforcement Learning via Slow Reinforcement Learning

Cited by in corpus (12)

• Optimizing Millions of Hyperparameters by Implicit Differentiation
• EvoGrad: Efficient Gradient-Based Meta-Learning and Hyperparameter Optimization
• Teaching with Commentaries
• A Value-Function-based Interior-point Method for Non-convex Bi-level Optimization
• Efficient hyperparameter optimization by way of PAC-Bayes bound minimization
• Shifting Transformation Learning for Out-of-Distribution Detection
• Contextual HyperNetworks for Novel Feature Adaptation
• Minimax Optimization with Smooth Algorithmic Adversaries
• Meta-learning One-class Classifiers with Eigenvalue Solvers for Supervised Anomaly Detection
• Learning Time-Invariant Reward Functions through Model-Based Inverse Reinforcement Learning
• On sensitivity of meta-learning to support data
• In-Loop Meta-Learning with Gradient-Alignment Reward