TAFE-Net: Task-Aware Feature Embeddings for Low Shot Learning

Abstract

Learning good feature embeddings for images often requires substantial training data. As a consequence, in settings where training data is limited (e.g., few-shot and zero-shot learning), we are typically forced to use a generic feature embedding across various tasks. Ideally, we want to construct feature embeddings that are tuned for the given task. In this work, we propose Task-Aware Feature Embedding Networks (TAFE-Nets) to learn how to adapt the image representation to a new task in a meta learning fashion. Our network is composed of a meta learner and a prediction network. Based on a task input, the meta learner generates parameters for the feature layers in the prediction network so that the feature embedding can be accurately adjusted for that task. We show that TAFE-Net is highly effective in generalizing to new tasks or concepts and evaluate the TAFE-Net on a range of benchmarks in zero-shot and few-shot learning. Our model matches or exceeds the state-of-the-art on all tasks. In particular, our approach improves the prediction accuracy of unseen attribute-object pairs by 4 to 15 points on the challenging visual attribute-object composition task.

Accepted at CVPR 2019

References in corpus (16)

• Adam: A Method for Stochastic Optimization
• Very Deep Convolutional Networks for Large-Scale Image Recognition
• Prototypical Networks for Few-shot Learning
• Deep Residual Learning for Image Recognition
• DeCAF: A Deep Convolutional Activation Feature for Generic Visual Recognition
• Matching Networks for One Shot Learning
• Fully Convolutional Networks for Semantic Segmentation
• Predicting Parameters in Deep Learning
• CyCADA: Cycle-Consistent Adversarial Domain Adaptation
• Rich feature hierarchies for accurate object detection and semantic segmentation
• Learning feed-forward one-shot learners
• Visual Relationship Detection with Language Priors
• SkipNet: Learning Dynamic Routing in Convolutional Networks
• BlockDrop: Dynamic Inference Paths in Residual Networks
• IDK Cascades: Fast Deep Learning by Learning not to Overthink
• Learning a Deep Embedding Model for Zero-Shot Learning

Cited by in corpus (6)

• HOI Analysis: Integrating and Decomposing Human-Object Interaction
• Dynamic Neural Networks: A Survey
• Meta-DETR: Image-Level Few-Shot Object Detection with Inter-Class Correlation Exploitation
• Task-Aware Feature Generation for Zero-Shot Compositional Learning
• Stochastic Conditional Generative Networks with Basis Decomposition
• Learning Single/Multi-Attribute of Object with Symmetry and Group