Zero-Shot Semantic Segmentation
arXiv:1906.00817
Abstract
Semantic segmentation models are limited in their ability to scale to large numbers of object classes. In this paper, we introduce the new task of zero-shot semantic segmentation: learning pixel-wise classifiers for never-seen object categories with zero training examples. To this end, we present a novel architecture, ZS3Net, combining a deep visual segmentation model with an approach to generate visual representations from semantic word embeddings. By this way, ZS3Net addresses pixel classification tasks where both seen and unseen categories are faced at test time (so called "generalized" zero-shot classification). Performance is further improved by a self-training step that relies on automatic pseudo-labeling of pixels from unseen classes. On the two standard segmentation datasets, Pascal-VOC and Pascal-Context, we propose zero-shot benchmarks and set competitive baselines. For complex scenes as ones in the Pascal-Context dataset, we extend our approach by using a graph-context encoding to fully leverage spatial context priors coming from class-wise segmentation maps.
arXiv commentNeurIPS 2019 (accepted)
References in corpus (17)
- Distributed Representations of Words and Phrases and their Compositionality
- Auto-Encoding Variational Bayes
- Semi-Supervised Classification with Graph Convolutional Networks
- Encoder-Decoder with Atrous Separable Convolution for Semantic Image Segmentation
- Zero-Shot Learning Through Cross-Modal Transfer
- Label-Embedding for Image Classification
- Zero-Shot Learning by Convex Combination of Semantic Embeddings
- DeepLab: Semantic Image Segmentation with Deep Convolutional Nets, Atrous Convolution, and Fully Connected CRFs
- Generative Moment Matching Networks
- Zero-Shot Learning -- A Comprehensive Evaluation of the Good, the Bad and the Ugly
- Context Encoding for Semantic Segmentation
- Zero-Shot Detection
- An Empirical Study and Analysis of Generalized Zero-Shot Learning for Object Recognition in the Wild
- Latent Embeddings for Zero-shot Classification
- Synthesized Classifiers for Zero-Shot Learning
- Transductive Unbiased Embedding for Zero-Shot Learning
- Zero-Shot Object Detection by Hybrid Region Embedding