Weighted Meta-Learning

Abstract

Meta-learning leverages related source tasks to learn an initialization that can be quickly fine-tuned to a target task with limited labeled examples. However, many popular meta-learning algorithms, such as model-agnostic meta-learning (MAML), only assume access to the target samples for fine-tuning. In this work, we provide a general framework for meta-learning based on weighting the loss of different source tasks, where the weights are allowed to depend on the target samples. In this general setting, we provide upper bounds on the distance of the weighted empirical risk of the source tasks and expected target risk in terms of an integral probability metric (IPM) and Rademacher complexity, which apply to a number of meta-learning settings including MAML and a weighted MAML variant. We then develop a learning algorithm based on minimizing the error bound with respect to an empirical IPM, including a weighted MAML algorithm, $\alpha$-MAML. Finally, we demonstrate empirically on several regression problems that our weighted meta-learning algorithm is able to find better initializations than uniformly-weighted meta-learning algorithms, such as MAML.

18 pages, 7 figures

References in corpus (22)

• Rejoinder to "Least angle regression" by Efron et al
• Least Angle Regression
• Discussion of "Least angle regression" by Efron et al
• Discussion of "Least angle regression" by Efron et al
• Discussion of "Least angle regression" by Efron et al
• Discussion of "Least angle regression" by Efron et al
• Discussion of "Least angle regression" by Efron et al
• Discussion of "Least angle regression" by Efron et al
• Discussion of "Least angle regression" by Efron et al
• Discussion of "Least angle regression" by Efron et al
• Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networks
• Prototypical Networks for Few-shot Learning
• Meta-SGD: Learning to Learn Quickly for Few-Shot Learning
• On First-Order Meta-Learning Algorithms
• Domain Adaptation: Learning Bounds and Algorithms
• Hierarchically Structured Meta-learning
• Online Meta-Learning
• On the Discrimination-Generalization Tradeoff in GANs
• Deep Online Learning via Meta-Learning: Continual Adaptation for Model-Based RL
• On the Convergence Theory of Gradient-Based Model-Agnostic Meta-Learning Algorithms
• Toward Multimodal Model-Agnostic Meta-Learning
• Learning GANs and Ensembles Using Discrepancy

Cited by in corpus (4)

• Task-Robust Model-Agnostic Meta-Learning
• Structured Prediction for Conditional Meta-Learning
• A Markov Decision Process Approach to Active Meta Learning
• For high-dimensional hierarchical models, consider exchangeability of effects across covariates instead of across datasets