Zero-Shot Learning from scratch (ZFS): leveraging local compositional representations
arXiv:2010.13320
Abstract
Zero-shot classification is a generalization task where no instance from the target classes is seen during training. To allow for test-time transfer, each class is annotated with semantic information, commonly in the form of attributes or text descriptions. While classical zero-shot learning does not explicitly forbid using information from other datasets, the approaches that achieve the best absolute performance on image benchmarks rely on features extracted from encoders pretrained on Imagenet. This approach relies on hyper-optimized Imagenet-relevant parameters from the supervised classification setting, entangling important questions about the suitability of those parameters and how they were learned with more fundamental questions about representation learning and generalization. To remove these distractors, we propose a more challenging setting: Zero-Shot Learning from scratch (ZFS), which explicitly forbids the use of encoders fine-tuned on other datasets. Our analysis on this setting highlights the importance of local information, and compositional representations.
arXiv commentICML 2019 Workshop on Understanding and Improving General-ization in Deep Learning, Long Beach, California, 2019 Spotlight presentation. arXiv admin note: text overlap with arXiv:1912.12179
References in corpus (15)
- Unsupervised Representation Learning with Deep Convolutional Generative Adversarial Networks
- Prototypical Networks for Few-shot Learning
- Self-Attention Generative Adversarial Networks
- Generative Adversarial Text to Image Synthesis
- Learning deep representations by mutual information estimation and maximization
- Matching Networks for One Shot Learning
- Learning Representations by Maximizing Mutual Information Across Views
- What makes ImageNet good for transfer learning?
- Zero-Shot Learning -- A Comprehensive Evaluation of the Good, the Bad and the Ugly
- Semantic Autoencoder for Zero-Shot Learning
- Bilinear Attention Networks
- An Analysis of Pre-Training on Object Detection
- Modular meta-learning
- Are Few-Shot Learning Benchmarks too Simple ? Solving them without Task Supervision at Test-Time
- Learning Semantically and Additively Compositional Distributional Representations