Contextualizing Enhances Gradient Based Meta Learning
arXiv:2007.10143
Abstract
Meta learning methods have found success when applied to few shot classification problems, in which they quickly adapt to a small number of labeled examples. Prototypical representations, each representing a particular class, have been of particular importance in this setting, as they provide a compact form to convey information learned from the labeled examples. However, these prototypes are just one method of representing this information, and they are narrow in their scope and ability to classify unseen examples. We propose the implementation of contextualizers, which are generalizable prototypes that adapt to given examples and play a larger role in classification for gradient-based models. We demonstrate how to equip meta learning methods with contextualizers and show that their use can significantly boost performance on a range of few shot learning datasets. We also present figures of merit demonstrating the potential benefits of contextualizers, along with analysis of how models make use of them. Our approach is particularly apt for low-data environments where it is difficult to update parameters without overfitting. Our implementation and instructions to reproduce the experiments are available at https://github.com/naveace/proto-context.
References in corpus (16)
- A Survey on Deep Learning in Medical Image Analysis
- Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networks
- Prototypical Networks for Few-shot Learning
- Fine-Grained Visual Classification of Aircraft
- Meta-SGD: Learning to Learn Quickly for Few-Shot Learning
- Meta-Learning for Semi-Supervised Few-Shot Classification
- A Neural Representation of Sketch Drawings
- Few-Shot Learning with Graph Neural Networks
- TADAM: Task dependent adaptive metric for improved few-shot learning
- Meta-Gradient Reinforcement Learning
- Meta Learning Shared Hierarchies
- Learned Optimizers that Scale and Generalize
- Meta-Learning and Universality: Deep Representations and Gradient Descent can Approximate any Learning Algorithm
- How to scale distributed deep learning?
- Alpha MAML: Adaptive Model-Agnostic Meta-Learning
- Few-shot Learning with Meta Metric Learners