Concept Learners for Few-Shot Learning
arXiv:2007.07375
Abstract
Developing algorithms that are able to generalize to a novel task given only a few labeled examples represents a fundamental challenge in closing the gap between machine- and human-level performance. The core of human cognition lies in the structured, reusable concepts that help us to rapidly adapt to new tasks and provide reasoning behind our decisions. However, existing meta-learning methods learn complex representations across prior labeled tasks without imposing any structure on the learned representations. Here we propose COMET, a meta-learning method that improves generalization ability by learning to learn along human-interpretable concept dimensions. Instead of learning a joint unstructured metric space, COMET learns mappings of high-level concepts into semi-structured metric spaces, and effectively combines the outputs of independent concept learners. We evaluate our model on few-shot tasks from diverse domains, including fine-grained image classification, document categorization and cell type annotation on a novel dataset from a biological domain developed in our work. COMET significantly outperforms strong meta-learning baselines, achieving 6-15% relative improvement on the most challenging 1-shot learning tasks, while unlike existing methods providing interpretations behind the model's predictions.
arXiv commentPublished at ICLR 2021
References in corpus (12)
- Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift
- Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networks
- Prototypical Networks for Few-shot Learning
- Axiomatic Attribution for Deep Networks
- SmoothGrad: removing noise by adding noise
- Towards Robust Interpretability with Self-Explaining Neural Networks
- Grad-CAM: Why did you say that?
- Deep Learning for Case-Based Reasoning through Prototypes: A Neural Network that Explains Its Predictions
- Cross Attention Network for Few-shot Classification
- Unsupervised Learning of Object Landmarks through Conditional Image Generation
- Infinite Mixture Prototypes for Few-Shot Learning
- Learning to Propagate for Graph Meta-Learning