Alpha MAML: Adaptive Model-Agnostic Meta-Learning

Abstract

Model-agnostic meta-learning (MAML) is a meta-learning technique to train a model on a multitude of learning tasks in a way that primes the model for few-shot learning of new tasks. The MAML algorithm performs well on few-shot learning problems in classification, regression, and fine-tuning of policy gradients in reinforcement learning, but comes with the need for costly hyperparameter tuning for training stability. We address this shortcoming by introducing an extension to MAML, called Alpha MAML, to incorporate an online hyperparameter adaptation scheme that eliminates the need to tune meta-learning and learning rates. Our results with the Omniglot database demonstrate a substantial reduction in the need to tune MAML training hyperparameters and improvement to training stability with less sensitivity to hyperparameter choice.

6th ICML Workshop on Automated Machine Learning (2019)

References in corpus (3)

• Practical Bayesian Optimization of Machine Learning Algorithms
• Theoretical Models of Learning to Learn
• Meta-SGD: Learning to Learn Quickly for Few-Shot Learning

Cited by in corpus (18)

• Personalized Federated Learning: A Meta-Learning Approach
• La-MAML: Look-ahead Meta Learning for Continual Learning
• Personalized Federated Learning using Hypernetworks
• Bayesian Active Meta-Learning for Few Pilot Demodulation and Equalization
• When MAML Can Adapt Fast and How to Assist When It Cannot
• Personalized Federated Learning with Gaussian Processes
• Domain Invariant Representation Learning with Domain Density Transformations
• Memory-Based Optimization Methods for Model-Agnostic Meta-Learning and Personalized Federated Learning
• Adaptive-Step Graph Meta-Learner for Few-Shot Graph Classification
• PAC-Bayes Bounds for Meta-learning with Data-Dependent Prior
• Embedding Adaptation is Still Needed for Few-Shot Learning
• Graceful Degradation and Related Fields
• Meta Adversarial Perturbations
• Contextualizing Enhances Gradient Based Meta Learning
• VIABLE: Fast Adaptation via Backpropagating Learned Loss
• Few Shot Learning With No Labels
• Is the Meta-Learning Idea Able to Improve the Generalization of Deep Neural Networks on the Standard Supervised Learning?
• Meta Learning in the Continuous Time Limit