Latent Embedding Feedback and Discriminative Features for Zero-Shot Classification
arXiv:2003.07833
Abstract
Zero-shot learning strives to classify unseen categories for which no data is available during training. In the generalized variant, the test samples can further belong to seen or unseen categories. The state-of-the-art relies on Generative Adversarial Networks that synthesize unseen class features by leveraging class-specific semantic embeddings. During training, they generate semantically consistent features, but discard this constraint during feature synthesis and classification. We propose to enforce semantic consistency at all stages of (generalized) zero-shot learning: training, feature synthesis and classification. We first introduce a feedback loop, from a semantic embedding decoder, that iteratively refines the generated features during both the training and feature synthesis stages. The synthesized features together with their corresponding latent embeddings from the decoder are then transformed into discriminative features and utilized during classification to reduce ambiguities among categories. Experiments on (generalized) zero-shot object and action classification reveal the benefit of semantic consistency and iterative feedback, outperforming existing methods on six zero-shot learning benchmarks. Source code at https://github.com/akshitac8/tfvaegan.
arXiv commentAccepted for publication at ECCV 2020
References in corpus (7)
- Distributed Representations of Words and Phrases and their Compositionality
- Auto-Encoding Variational Bayes
- UCF101: A Dataset of 101 Human Actions Classes From Videos in The Wild
- Transductive Multi-view Zero-Shot Learning
- Zero Shot Recognition with Unreliable Attributes
- Leveraging the Invariant Side of Generative Zero-Shot Learning
- Towards Universal Representation for Unseen Action Recognition
Cited by in corpus (6)
- Meta-Learned Attribute Self-Gating for Continual Generalized Zero-Shot Learning
- CIZSL++: Creativity Inspired Generative Zero-Shot Learning
- End-to-end Generative Zero-shot Learning via Few-shot Learning
- DFS: A Diverse Feature Synthesis Model for Generalized Zero-Shot Learning
- Mitigating Generation Shifts for Generalized Zero-Shot Learning
- Using Fictitious Class Representations to Boost Discriminative Zero-Shot Learners