TADAM: Task dependent adaptive metric for improved few-shot learning

Abstract

Few-shot learning has become essential for producing models that generalize from few examples. In this work, we identify that metric scaling and metric task conditioning are important to improve the performance of few-shot algorithms. Our analysis reveals that simple metric scaling completely changes the nature of few-shot algorithm parameter updates. Metric scaling provides improvements up to 14% in accuracy for certain metrics on the mini-Imagenet 5-way 5-shot classification task. We further propose a simple and effective way of conditioning a learner on the task sample set, resulting in learning a task-dependent metric space. Moreover, we propose and empirically test a practical end-to-end optimization procedure based on auxiliary task co-training to learn a task-dependent metric space. The resulting few-shot learning model based on the task-dependent scaled metric achieves state of the art on mini-Imagenet. We confirm these results on another few-shot dataset that we introduce in this paper based on CIFAR100. Our code is publicly available at https://github.com/ElementAI/TADAM.

References in corpus (6)

• Distilling the Knowledge in a Neural Network
• A Learned Representation For Artistic Style
• Meta-Learning for Semi-Supervised Few-Shot Classification
• FiLM: Visual Reasoning with a General Conditioning Layer
• Discriminative k-shot learning using probabilistic models
• Deep Prior

Cited by in corpus (31)

• SimpleShot: Revisiting Nearest-Neighbor Classification for Few-Shot Learning
• Meta-Learning with Latent Embedding Optimization
• Finding Task-Relevant Features for Few-Shot Learning by Category Traversal
• A Comprehensive Overview and Survey of Recent Advances in Meta-Learning
• Prior-Knowledge and Attention-based Meta-Learning for Few-Shot Learning
• A Comparative Review of Recent Few-Shot Object Detection Algorithms
• Towards Gradient-based Bilevel Optimization with Non-convex Followers and Beyond
• Learning Adaptive Classifiers Synthesis for Generalized Few-Shot Learning
• Reconciling meta-learning and continual learning with online mixtures of tasks
• L2AE-D: Learning to Aggregate Embeddings for Few-shot Learning with Meta-level Dropout
• Temperature as Uncertainty in Contrastive Learning
• Is Support Set Diversity Necessary for Meta-Learning?
• XtarNet: Learning to Extract Task-Adaptive Representation for Incremental Few-Shot Learning
• Meta-Learning with Adaptive Hyperparameters
• Texture Bias Of CNNs Limits Few-Shot Classification Performance
• Looking back to lower-level information in few-shot learning
• Adaptive Cross-Modal Few-Shot Learning
• Object-aware Long-short-range Spatial Alignment for Few-Shot Fine-Grained Image Classification
• Sign-MAML: Efficient Model-Agnostic Meta-Learning by SignSGD
• FedSC: Federated Learning with Semantic-Aware Collaboration
• Contextualizing Enhances Gradient Based Meta Learning
• Efficient Gradient Approximation Method for Constrained Bilevel Optimization
• Decoder Choice Network for Meta-Learning
• Long-term Cross Adversarial Training: A Robust Meta-learning Method for Few-shot Classification Tasks
• ST-MAML: A Stochastic-Task based Method for Task-Heterogeneous Meta-Learning
• Meta-Learner with Linear Nulling
• Task Affinity with Maximum Bipartite Matching in Few-Shot Learning
• Learning by Examples Based on Multi-level Optimization
• Meta-Learning with Task-Adaptive Loss Function for Few-Shot Learning
• On sensitivity of meta-learning to support data
• Uniform Sampling over Episode Difficulty