Weakly-supervised Compositional FeatureAggregation for Few-shot Recognition

Abstract

Learning from a few examples is a challenging task for machine learning. While recent progress has been made for this problem, most of the existing methods ignore the compositionality in visual concept representation (e.g. objects are built from parts or composed of semantic attributes), which is key to the human ability to easily learn from a small number of examples. To enhance the few-shot learning models with compositionality, in this paper we present the simple yet powerful Compositional Feature Aggregation (CFA) module as a weakly-supervised regularization for deep networks. Given the deep feature maps extracted from the input, our CFA module first disentangles the feature space into disjoint semantic subspaces that model different attributes, and then bilinearly aggregates the local features within each of these subspaces. CFA explicitly regularizes the representation with both semantic and spatial compositionality to produce discriminative representations for few-shot recognition tasks. Moreover, our method does not need any supervision for attributes and object parts during training, thus can be conveniently plugged into existing models for end-to-end optimization while keeping the model size and computation cost nearly the same. Extensive experiments on few-shot image classification and action recognition tasks demonstrate that our method provides substantial improvements over recent state-of-the-art methods.

References in corpus (13)

• Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networks
• Two-Stream Convolutional Networks for Action Recognition in Videos
• Prototypical Networks for Few-shot Learning
• The Kinetics Human Action Video Dataset
• Matching Networks for One Shot Learning
• A Simple Neural Attentive Meta-Learner
• Data Augmentation Generative Adversarial Networks
• Generalizing from a Few Examples: A Survey on Few-Shot Learning
• Few-Shot Learning with Graph Neural Networks
• Few-Shot Learning with Metric-Agnostic Conditional Embeddings
• Few-shot Object Detection via Feature Reweighting
• Similarity R-C3D for Few-shot Temporal Activity Detection
• Adaptive Cross-Modal Few-Shot Learning

Cited by in corpus (6)

• TOAN: Target-Oriented Alignment Network for Fine-Grained Image Categorization with Few Labeled Samples
• Multi-modality Latent Interaction Network for Visual Question Answering
• Prototype Completion with Primitive Knowledge for Few-Shot Learning
• Compositional Few-Shot Recognition with Primitive Discovery and Enhancing
• Revisiting Mid-Level Patterns for Cross-Domain Few-Shot Recognition
• Depth Guided Adaptive Meta-Fusion Network for Few-shot Video Recognition