Learning Joint Feature Adaptation for Zero-Shot Recognition

Abstract

Zero-shot recognition (ZSR) aims to recognize target-domain data instances of unseen classes based on the models learned from associated pairs of seen-class source and target domain data. One of the key challenges in ZSR is the relative scarcity of source-domain features (e.g. one feature vector per class), which do not fully account for wide variability in target-domain instances. In this paper we propose a novel framework of learning data-dependent feature transforms for scoring similarity between an arbitrary pair of source and target data instances to account for the wide variability in target domain. Our proposed approach is based on optimizing over a parameterized family of local feature displacements that maximize the source-target adaptive similarity functions. Accordingly we propose formulating zero-shot learning (ZSL) using latent structural SVMs to learn our similarity functions from training data. As demonstration we design a specific algorithm under the proposed framework involving bilinear similarity functions and regularized least squares as penalties for feature displacement. We test our approach on several benchmark datasets for ZSR and show significant improvement over the state-of-the-art. For instance, on aP&Y dataset we can achieve 80.89% in terms of recognition accuracy, outperforming the state-of-the-art by 11.15%.

References in corpus (10)

• Very Deep Convolutional Networks for Large-Scale Image Recognition
• MatConvNet - Convolutional Neural Networks for MATLAB
• Transductive Multi-view Zero-Shot Learning
• Predicting Deep Zero-Shot Convolutional Neural Networks using Textual Descriptions
• Zero-Shot Learning via Semantic Similarity Embedding
• An Empirical Study and Analysis of Generalized Zero-Shot Learning for Object Recognition in the Wild
• Online but Accurate Inference for Latent Variable Models with Local Gibbs Sampling
• Recovering the Missing Link: Predicting Class-Attribute Associations for Unsupervised Zero-Shot Learning
• Marginal Structured SVM with Hidden Variables
• Zero-Shot Learning via Joint Latent Similarity Embedding

Cited by in corpus (6)

• Zero-Shot Learning via Class-Conditioned Deep Generative Models
• A Meta-Learning Framework for Generalized Zero-Shot Learning
• Meta-Learned Attribute Self-Gating for Continual Generalized Zero-Shot Learning
• Structure-Aware Feature Generation for Zero-Shot Learning
• Learning Clusterable Visual Features for Zero-Shot Recognition
• Zero-Shot Recognition via Optimal Transport