Meta-Dataset: A Dataset of Datasets for Learning to Learn from Few Examples
arXiv:1903.03096
Abstract
Few-shot classification refers to learning a classifier for new classes given only a few examples. While a plethora of models have emerged to tackle it, we find the procedure and datasets that are used to assess their progress lacking. To address this limitation, we propose Meta-Dataset: a new benchmark for training and evaluating models that is large-scale, consists of diverse datasets, and presents more realistic tasks. We experiment with popular baselines and meta-learners on Meta-Dataset, along with a competitive method that we propose. We analyze performance as a function of various characteristics of test tasks and examine the models' ability to leverage diverse training sources for improving their generalization. We also propose a new set of baselines for quantifying the benefit of meta-learning in Meta-Dataset. Our extensive experimentation has uncovered important research challenges and we hope to inspire work in these directions.
arXiv commentCode available at https://github.com/google-research/meta-dataset
References in corpus (4)
Cited by in corpus (10)
- Intriguing Properties of Vision Transformers
- Automated Human Cell Classification in Sparse Datasets using Few-Shot Learning
- Meta-Learning GNN Initializations for Low-Resource Molecular Property Prediction
- On Episodes, Prototypical Networks, and Few-shot Learning
- Learning Adaptive Classifiers Synthesis for Generalized Few-Shot Learning
- Modularity in Deep Learning: A Survey
- Meta Learning Backpropagation And Improving It
- An empirical study of pretrained representations for few-shot classification
- Expanding continual few-shot learning benchmarks to include recognition of specific instances
- MapRE: An Effective Semantic Mapping Approach for Low-resource Relation Extraction