Adaptive Confidence Smoothing for Generalized Zero-Shot Learning

Abstract

Generalized zero-shot learning (GZSL) is the problem of learning a classifier where some classes have samples and others are learned from side information, like semantic attributes or text description, in a zero-shot learning fashion (ZSL). Training a single model that operates in these two regimes simultaneously is challenging. Here we describe a probabilistic approach that breaks the model into three modular components, and then combines them in a consistent way. Specifically, our model consists of three classifiers: A "gating" model that makes soft decisions if a sample is from a "seen" class, and two experts: a ZSL expert, and an expert model for seen classes. We address two main difficulties in this approach: How to provide an accurate estimate of the gating probability without any training samples for unseen classes; and how to use expert predictions when it observes samples outside of its domain. The key insight to our approach is to pass information between the three models to improve each one's accuracy, while maintaining the modular structure. We test our approach, adaptive confidence smoothing (COSMO), on four standard GZSL benchmark datasets and find that it largely outperforms state-of-the-art GZSL models. COSMO is also the first model that closes the gap and surpasses the performance of generative models for GZSL, even-though it is a light-weight model that is much easier to train and tune. Notably, COSMO offers a new view for developing zero-shot models. Thanks to COSMO's modular structure, instead of trying to perform well both on seen and on unseen classes, models can focus on accurate classification of unseen classes, and later consider seen class models.

arXiv comment(1) Accepted to CVPR 2019. (2) Previous title was "Domain-Aware Generalized Zero-Shot Learning". (3) This arxiv version is as the CVPR final version with the following modifications: (a) corrected typos found in Table 3 (b) updated "Related Work" with [52, 10, 20] (c) add a paragraph to the abstract (d) add a probabilistic explanation for the smoothing term

References in corpus (16)

• Scikit-learn: Machine Learning in Python
• Distilling the Knowledge in a Neural Network
• A Baseline for Detecting Misclassified and Out-of-Distribution Examples in Neural Networks
• Zero-Shot Learning Through Cross-Modal Transfer
• Outrageously Large Neural Networks: The Sparsely-Gated Mixture-of-Experts Layer
• Zero Shot Recognition with Unreliable Attributes
• Semantic Autoencoder for Zero-Shot Learning
• Preserving Semantic Relations for Zero-Shot Learning
• An Empirical Study and Analysis of Generalized Zero-Shot Learning for Object Recognition in the Wild
• Learning to generalize to new compositions in image understanding
• Recent Advances in Zero-shot Recognition
• Domain-Invariant Projection Learning for Zero-Shot Recognition
• Probabilistic AND-OR Attribute Grouping for Zero-Shot Learning
• Out-of-Distribution Detection using Multiple Semantic Label Representations
• Learning Robust Visual-Semantic Embeddings
• Semantically Consistent Regularization for Zero-Shot Recognition

Cited by in corpus (1)

• Learning Classifiers for Domain Adaptation, Zero and Few-Shot Recognition Based on Learning Latent Semantic Parts