Latent Embeddings for Zero-shot Classification

Abstract

We present a novel latent embedding model for learning a compatibility function between image and class embeddings, in the context of zero-shot classification. The proposed method augments the state-of-the-art bilinear compatibility model by incorporating latent variables. Instead of learning a single bilinear map, it learns a collection of maps with the selection, of which map to use, being a latent variable for the current image-class pair. We train the model with a ranking based objective function which penalizes incorrect rankings of the true class for a given image. We empirically demonstrate that our model improves the state-of-the-art for various class embeddings consistently on three challenging publicly available datasets for the zero-shot setting. Moreover, our method leads to visually highly interpretable results with clear clusters of different fine-grained object properties that correspond to different latent variable maps.

arXiv comment2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR)

References in corpus (3)

• Going Deeper with Convolutions
• Learning Hypergraph-regularized Attribute Predictors
• Zero-Shot Learning via Joint Latent Similarity Embedding

Cited by in corpus (18)

• Zero-Shot Learning -- The Good, the Bad and the Ugly
• Learning to generalize to new compositions in image understanding
• Zero-Shot Learning by Generating Pseudo Feature Representations
• Semi-supervised Zero-Shot Learning by a Clustering-based Approach
• Multi-Head Self-Attention via Vision Transformer for Zero-Shot Learning
• Mitigating the Hubness Problem for Zero-Shot Learning of 3D Objects
• Learning the Redundancy-free Features for Generalized Zero-Shot Object Recognition
• Zero-Shot Semantic Segmentation
• Zero-Shot Visual Recognition via Bidirectional Latent Embedding
• Domain segmentation and adjustment for generalized zero-shot learning
• Classifier and Exemplar Synthesis for Zero-Shot Learning
• Weak Novel Categories without Tears: A Survey on Weak-Shot Learning
• Hard Negative Mining for Metric Learning Based Zero-Shot Classification
• Mitigating Generation Shifts for Generalized Zero-Shot Learning
• Learning Clusterable Visual Features for Zero-Shot Recognition
• Zero-Shot Audio Classification Based on Class Label Embeddings
• Beyond Attributes: Adversarial Erasing Embedding Network for Zero-shot Learning
• Zero-Shot Learning from Adversarial Feature Residual to Compact Visual Feature