Semantically Consistent Regularization for Zero-Shot Recognition

Abstract

The role of semantics in zero-shot learning is considered. The effectiveness of previous approaches is analyzed according to the form of supervision provided. While some learn semantics independently, others only supervise the semantic subspace explained by training classes. Thus, the former is able to constrain the whole space but lacks the ability to model semantic correlations. The latter addresses this issue but leaves part of the semantic space unsupervised. This complementarity is exploited in a new convolutional neural network (CNN) framework, which proposes the use of semantics as constraints for recognition.Although a CNN trained for classification has no transfer ability, this can be encouraged by learning an hidden semantic layer together with a semantic code for classification. Two forms of semantic constraints are then introduced. The first is a loss-based regularizer that introduces a generalization constraint on each semantic predictor. The second is a codeword regularizer that favors semantic-to-class mappings consistent with prior semantic knowledge while allowing these to be learned from data. Significant improvements over the state-of-the-art are achieved on several datasets.

Accepted to CVPR 2017

References in corpus (5)

• Very Deep Convolutional Networks for Large-Scale Image Recognition
• Caffe: Convolutional Architecture for Fast Feature Embedding
• Going Deeper with Convolutions
• Transductive Multi-view Zero-Shot Learning
• Recovering the Missing Link: Predicting Class-Attribute Associations for Unsupervised Zero-Shot Learning

Cited by in corpus (12)

• Preserving Semantic Relations for Zero-Shot Learning
• Discriminative Learning of Latent Features for Zero-Shot Recognition
• Transductive Unbiased Embedding for Zero-Shot Learning
• Stacked Semantic-Guided Attention Model for Fine-Grained Zero-Shot Learning
• Zero-Shot Visual Recognition using Semantics-Preserving Adversarial Embedding Networks
• Learning Class Prototypes via Structure Alignment for Zero-Shot Recognition
• Region Semantically Aligned Network for Zero-Shot Learning
• Adaptive Confidence Smoothing for Generalized Zero-Shot Learning
• Solving Long-tailed Recognition with Deep Realistic Taxonomic Classifier
• Bi-Adversarial Auto-Encoder for Zero-Shot Learning
• Global Semantic Consistency for Zero-Shot Learning
• Beyond Attributes: Adversarial Erasing Embedding Network for Zero-shot Learning