Few-Shot Learning via Embedding Adaptation with Set-to-Set Functions

Abstract

Learning with limited data is a key challenge for visual recognition. Many few-shot learning methods address this challenge by learning an instance embedding function from seen classes and apply the function to instances from unseen classes with limited labels. This style of transfer learning is task-agnostic: the embedding function is not learned optimally discriminative with respect to the unseen classes, where discerning among them leads to the target task. In this paper, we propose a novel approach to adapt the instance embeddings to the target classification task with a set-to-set function, yielding embeddings that are task-specific and are discriminative. We empirically investigated various instantiations of such set-to-set functions and observed the Transformer is most effective -- as it naturally satisfies key properties of our desired model. We denote this model as FEAT (few-shot embedding adaptation w/ Transformer) and validate it on both the standard few-shot classification benchmark and four extended few-shot learning settings with essential use cases, i.e., cross-domain, transductive, generalized few-shot learning, and low-shot learning. It archived consistent improvements over baseline models as well as previous methods and established the new state-of-the-art results on two benchmarks.

Accepted by CVPR 2020; The code is available at https://github.com/Sha-Lab/FEAT

References in corpus (13)

• Very Deep Convolutional Networks for Large-Scale Image Recognition
• Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift
• A Structured Self-attentive Sentence Embedding
• Learning to Propagate Labels: Transductive Propagation Network for Few-shot Learning
• DropBlock: A regularization method for convolutional networks
• Layer Normalization
• Universal Transformers
• SimpleShot: Revisiting Nearest-Neighbor Classification for Few-Shot Learning
• Unsupervised Learning via Meta-Learning
• Piecewise classifier mappings: Learning fine-grained learners for novel categories with few examples
• Adapted Deep Embeddings: A Synthesis of Methods for $k$-Shot Inductive Transfer Learning
• Transductive Episodic-Wise Adaptive Metric for Few-Shot Learning
• Low-shot Learning via Covariance-Preserving Adversarial Augmentation Networks

Cited by in corpus (10)

• SimpleShot: Revisiting Nearest-Neighbor Classification for Few-Shot Learning
• Leveraging the Feature Distribution in Transfer-based Few-Shot Learning
• DPGN: Distribution Propagation Graph Network for Few-shot Learning
• Revisiting Meta-Learning as Supervised Learning
• Graph-based Interpolation of Feature Vectors for Accurate Few-Shot Classification
• Compositional Few-Shot Recognition with Primitive Discovery and Enhancing
• Object-aware Long-short-range Spatial Alignment for Few-Shot Fine-Grained Image Classification
• Revisiting Mid-Level Patterns for Cross-Domain Few-Shot Recognition
• ReMP: Rectified Metric Propagation for Few-Shot Learning
• Learning Classifiers for Domain Adaptation, Zero and Few-Shot Recognition Based on Learning Latent Semantic Parts