Isometric Propagation Network for Generalized Zero-shot Learning

Abstract

Zero-shot learning (ZSL) aims to classify images of an unseen class only based on a few attributes describing that class but no access to any training sample. A popular strategy is to learn a mapping between the semantic space of class attributes and the visual space of images based on the seen classes and their data. Thus, an unseen class image can be ideally mapped to its corresponding class attributes. The key challenge is how to align the representations in the two spaces. For most ZSL settings, the attributes for each seen/unseen class are only represented by a vector while the seen-class data provide much more information. Thus, the imbalanced supervision from the semantic and the visual space can make the learned mapping easily overfitting to the seen classes. To resolve this problem, we propose Isometric Propagation Network (IPN), which learns to strengthen the relation between classes within each space and align the class dependency in the two spaces. Specifically, IPN learns to propagate the class representations on an auto-generated graph within each space. In contrast to only aligning the resulted static representation, we regularize the two dynamic propagation procedures to be isometric in terms of the two graphs' edge weights per step by minimizing a consistency loss between them. IPN achieves state-of-the-art performance on three popular ZSL benchmarks. To evaluate the generalization capability of IPN, we further build two larger benchmarks with more diverse unseen classes and demonstrate the advantages of IPN on them.

Accepted by ICLR 2021

References in corpus (9)

• Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networks
• Prototypical Networks for Few-shot Learning
• ConceptNet 5.5: An Open Multilingual Graph of General Knowledge
• Zero-Shot Learning Through Cross-Modal Transfer
• Label-Embedding for Image Classification
• Zero-Shot Learning by Convex Combination of Semantic Embeddings
• NATS-Bench: Benchmarking NAS Algorithms for Architecture Topology and Size
• Learning to Propagate for Graph Meta-Learning
• Zero-Shot Recognition using Dual Visual-Semantic Mapping Paths

Cited by in corpus (5)

• A survey on Semi-, Self- and Unsupervised Learning for Image Classification
• Dual Progressive Prototype Network for Generalized Zero-Shot Learning
• Zero-Shot Learning with Common Sense Knowledge Graphs
• Meta-Learned Attribute Self-Gating for Continual Generalized Zero-Shot Learning
• Meta Two-Sample Testing: Learning Kernels for Testing with Limited Data