Revisiting Meta-Learning as Supervised Learning
arXiv:2002.00573
Abstract
Recent years have witnessed an abundance of new publications and approaches on meta-learning. This community-wide enthusiasm has sparked great insights but has also created a plethora of seemingly different frameworks, which can be hard to compare and evaluate. In this paper, we aim to provide a principled, unifying framework by revisiting and strengthening the connection between meta-learning and traditional supervised learning. By treating pairs of task-specific data sets and target models as (feature, label) samples, we can reduce many meta-learning algorithms to instances of supervised learning. This view not only unifies meta-learning into an intuitive and practical framework but also allows us to transfer insights from supervised learning directly to improve meta-learning. For example, we obtain a better understanding of generalization properties, and we can readily transfer well-understood techniques, such as model ensemble, pre-training, joint training, data augmentation, and even nearest neighbor based methods. We provide an intuitive analogy of these methods in the context of meta-learning and show that they give rise to significant improvements in model performance on few-shot learning.
An extended version of the paper titled "A Meta Understanding of Meta-Learning" presented in ICML 2019 Workshop on Adaptive and Multitask Learning: Algorithms & Systems
References in corpus (20)
- Prototypical Networks for Few-shot Learning
- How transferable are features in deep neural networks?
- Neural Architecture Search: A Survey
- Theoretical Models of Learning to Learn
- Fine-Grained Visual Classification of Aircraft
- A Simple Neural Attentive Meta-Learner
- Meta-SGD: Learning to Learn Quickly for Few-Shot Learning
- RL: Fast Reinforcement Learning via Slow Reinforcement Learning
- Learning to reinforcement learn
- Meta-Learning: A Survey
- Learning to Compose Domain-Specific Transformations for Data Augmentation
- Unsupervised Learning via Meta-Learning
- The Benefit of Multitask Representation Learning
- Meta-Learning Update Rules for Unsupervised Representation Learning
- Few-Shot Learning via Embedding Adaptation with Set-to-Set Functions
- Feature-Critic Networks for Heterogeneous Domain Generalization
- Meta-Learning Transferable Active Learning Policies by Deep Reinforcement Learning
- Learning Algorithms for Active Learning
- Supervising Unsupervised Learning
- Natural Language to Structured Query Generation via Meta-Learning