Leveraging the Invariant Side of Generative Zero-Shot Learning

Abstract

Conventional zero-shot learning (ZSL) methods generally learn an embedding, e.g., visual-semantic mapping, to handle the unseen visual samples via an indirect manner. In this paper, we take the advantage of generative adversarial networks (GANs) and propose a novel method, named leveraging invariant side GAN (LisGAN), which can directly generate the unseen features from random noises which are conditioned by the semantic descriptions. Specifically, we train a conditional Wasserstein GANs in which the generator synthesizes fake unseen features from noises and the discriminator distinguishes the fake from real via a minimax game. Considering that one semantic description can correspond to various synthesized visual samples, and the semantic description, figuratively, is the soul of the generated features, we introduce soul samples as the invariant side of generative zero-shot learning in this paper. A soul sample is the meta-representation of one class. It visualizes the most semantically-meaningful aspects of each sample in the same category. We regularize that each generated sample (the varying side of generative ZSL) should be close to at least one soul sample (the invariant side) which has the same class label with it. At the zero-shot recognition stage, we propose to use two classifiers, which are deployed in a cascade way, to achieve a coarse-to-fine result. Experiments on five popular benchmarks verify that our proposed approach can outperform state-of-the-art methods with significant improvements.

arXiv commentAccepted in CVPR 2019

References in corpus (6)

• Conditional Generative Adversarial Nets
• Prototypical Networks for Few-shot Learning
• Label-Embedding for Image Classification
• Zero-Shot Learning by Convex Combination of Semantic Embeddings
• Semantic Autoencoder for Zero-Shot Learning
• Zero-Shot Learning -- The Good, the Bad and the Ugly

Cited by in corpus (38)

• Latent Embedding Feedback and Discriminative Features for Zero-Shot Classification
• Fine-Grained Zero-Shot Learning with DNA as Side Information
• Online Lifelong Generalized Zero-Shot Learning
• Episode-based Prototype Generating Network for Zero-Shot Learning
• Learning the Redundancy-free Features for Generalized Zero-Shot Object Recognition
• Zero-Shot Learning with Common Sense Knowledge Graphs
• A Meta-Learning Framework for Generalized Zero-Shot Learning
• Generative Adversarial Zero-shot Learning via Knowledge Graphs
• FREE: Feature Refinement for Generalized Zero-Shot Learning
• Generalized Continual Zero-Shot Learning
• Attribute Prototype Network for Zero-Shot Learning
• Visual Space Optimization for Zero-shot Learning
• Domain segmentation and adjustment for generalized zero-shot learning
• Zero-Shot Learning with Knowledge Enhanced Visual Semantic Embeddings
• Hardness Sampling for Self-Training Based Transductive Zero-Shot Learning
• Information Bottleneck Constrained Latent Bidirectional Embedding for Zero-Shot Learning
• Multi-Knowledge Fusion for New Feature Generation in Generalized Zero-Shot Learning
• Cycle-consistent Conditional Adversarial Transfer Networks
• Rethinking Generative Zero-Shot Learning: An Ensemble Learning Perspective for Recognising Visual Patches
• GTNet: Generative Transfer Network for Zero-Shot Object Detection
• Alleviating Feature Confusion for Generative Zero-shot Learning
• Entropy-Based Uncertainty Calibration for Generalized Zero-Shot Learning
• Structured Latent Embeddings for Recognizing Unseen Classes in Unseen Domains
• Learning without Seeing nor Knowing: Towards Open Zero-Shot Learning
• Multi-Feature Discrete Collaborative Filtering for Fast Cold-start Recommendation
• Bi-Semantic Reconstructing Generative Network for Zero-shot Learning
• OntoZSL: Ontology-enhanced Zero-shot Learning
• Improving Generalized Zero-Shot Learning by Semantic Discriminator
• End-to-end Generative Zero-shot Learning via Few-shot Learning
• Bias-Awareness for Zero-Shot Learning the Seen and Unseen
• Towards Visual Explainable Active Learning for Zero-Shot Classification
• Transductive Zero-Shot Learning by Decoupled Feature Generation
• DFS: A Diverse Feature Synthesis Model for Generalized Zero-Shot Learning
• Learning Clusterable Visual Features for Zero-Shot Recognition
• Toward Zero-Shot Unsupervised Image-to-Image Translation
• Mitigating Generation Shifts for Generalized Zero-Shot Learning
• Visual and Semantic Prototypes-Jointly Guided CNN for Generalized Zero-shot Learning
• Zero-Shot Learning from Adversarial Feature Residual to Compact Visual Feature