Task-Aware Feature Generation for Zero-Shot Compositional Learning
arXiv:1906.04854
Abstract
Visual concepts (e.g., red apple, big elephant) are often semantically compositional and each element of the compositions can be reused to construct novel concepts (e.g., red elephant). Compositional feature synthesis, which generates image feature distributions exploiting the semantic compositionality, is a promising approach to sample-efficient model generalization. In this work, we propose a task-aware feature generation (TFG) framework for compositional learning, which generates features of novel visual concepts by transferring knowledge from previously seen concepts. These synthetic features are then used to train a classifier to recognize novel concepts in a zero-shot manner. Our novel TFG design injects task-conditioned noise layer-by-layer, producing task-relevant variation at each level. We find the proposed generator design improves classification accuracy and sample efficiency. Our model establishes a new state of the art on three zero-shot compositional learning (ZSCL) benchmarks, outperforming the previous discriminative models by a large margin. Our model improves the performance of the prior arts by over 2x in the generalized ZSCL setting.
arXiv comment17 pages, 9 figures; substantial content updates with additional experiments
References in corpus (14)
- Adam: A Method for Stochastic Optimization
- Conditional Generative Adversarial Nets
- Prototypical Networks for Few-shot Learning
- Large Scale GAN Training for High Fidelity Natural Image Synthesis
- Self-Attention Generative Adversarial Networks
- Generative Adversarial Text to Image Synthesis
- Matching Networks for One Shot Learning
- Which Training Methods for GANs do actually Converge?
- cGANs with Projection Discriminator
- Semantic Jitter: Dense Supervision for Visual Comparisons via Synthetic Images
- Dynamic Few-Shot Visual Learning without Forgetting
- Low-shot Learning via Covariance-Preserving Adversarial Augmentation Networks
- Multi-modal Cycle-consistent Generalized Zero-Shot Learning
- TAFE-Net: Task-Aware Feature Embeddings for Low Shot Learning