Generalized Inner Loop Meta-Learning

Abstract

Many (but not all) approaches self-qualifying as "meta-learning" in deep learning and reinforcement learning fit a common pattern of approximating the solution to a nested optimization problem. In this paper, we give a formalization of this shared pattern, which we call GIMLI, prove its general requirements, and derive a general-purpose algorithm for implementing similar approaches. Based on this analysis and algorithm, we describe a library of our design, higher, which we share with the community to assist and enable future research into these kinds of meta-learning approaches. We end the paper by showcasing the practical applications of this framework and library through illustrative experiments and ablation studies which they facilitate.

17 pages, 3 figures, 1 algorithm

References in corpus (9)

• Adam: A Method for Stochastic Optimization
• Very Deep Convolutional Networks for Large-Scale Image Recognition
• Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networks
• Practical Bayesian Optimization of Machine Learning Algorithms
• RL${}^2$: Fast Reinforcement Learning via Slow Reinforcement Learning
• Meta-Learning with Latent Embedding Optimization
• Bilevel Programming for Hyperparameter Optimization and Meta-Learning
• Scalable Gradient-Based Tuning of Continuous Regularization Hyperparameters
• How to train your MAML

Cited by in corpus (38)

• Balanced Meta-Softmax for Long-Tailed Visual Recognition
• A Survey of Contextual Optimization Methods for Decision Making under Uncertainty
• Meta-learning PINN loss functions
• Automatic Data Augmentation for Generalization in Deep Reinforcement Learning
• Optimizing Millions of Hyperparameters by Implicit Differentiation
• A Meta-Learning Approach for Training Explainable Graph Neural Networks
• Learning Neural Event Functions for Ordinary Differential Equations
• Meta-Learning Symmetries by Reparameterization
• Deep Transformers with Latent Depth
• Balanced softmax cross-entropy for incremental learning with and without memory
• MobilePhys: Personalized Mobile Camera-Based Contactless Physiological Sensing
• Learning Optimal Representations with the Decodable Information Bottleneck
• On Episodes, Prototypical Networks, and Few-shot Learning
• Offline Meta-Reinforcement Learning with Advantage Weighting
• MetaPhys: Few-Shot Adaptation for Non-Contact Physiological Measurement
• Model-Based Inverse Reinforcement Learning from Visual Demonstrations
• Learning State-Dependent Losses for Inverse Dynamics Learning
• EvoGrad: Efficient Gradient-Based Meta-Learning and Hyperparameter Optimization
• Tailoring: encoding inductive biases by optimizing unsupervised objectives at prediction time
• A Value-Function-based Interior-point Method for Non-convex Bi-level Optimization
• Teaching with Commentaries
• Efficient hyperparameter optimization by way of PAC-Bayes bound minimization
• gradSLAM: Automagically differentiable SLAM
• Adversarially Regularized Policy Learning Guided by Trajectory Optimization
• Global Convergence and Generalization Bound of Gradient-Based Meta-Learning with Deep Neural Nets
• OmniPrint: A Configurable Printed Character Synthesizer
• Shifting Transformation Learning for Out-of-Distribution Detection
• Few-shot Learning for Spatial Regression
• Contextual HyperNetworks for Novel Feature Adaptation
• Fast Model Editing at Scale
• Value-Function-based Sequential Minimization for Bi-level Optimization
• Minimax Optimization with Smooth Algorithmic Adversaries
• Meta-learning One-class Classifiers with Eigenvalue Solvers for Supervised Anomaly Detection
• On sensitivity of meta-learning to support data
• In-Loop Meta-Learning with Gradient-Alignment Reward
• Gradient scarcity with Bilevel Optimization for Graph Learning
• Adversarial Regularization as Stackelberg Game: An Unrolled Optimization Approach
• Learning Time-Invariant Reward Functions through Model-Based Inverse Reinforcement Learning