Universal Semi-Supervised Semantic Segmentation
arXiv:1811.10323
Abstract
In recent years, the need for semantic segmentation has arisen across several different applications and environments. However, the expense and redundancy of annotation often limits the quantity of labels available for training in any domain, while deployment is easier if a single model works well across domains. In this paper, we pose the novel problem of universal semi-supervised semantic segmentation and propose a solution framework, to meet the dual needs of lower annotation and deployment costs. In contrast to counterpoints such as fine tuning, joint training or unsupervised domain adaptation, universal semi-supervised segmentation ensures that across all domains: (i) a single model is deployed, (ii) unlabeled data is used, (iii) performance is improved, (iv) only a few labels are needed and (v) label spaces may differ. To address this, we minimize supervised as well as within and cross-domain unsupervised losses, introducing a novel feature alignment objective based on pixel-aware entropy regularization for the latter. We demonstrate quantitative advantages over other approaches on several combinations of segmentation datasets across different geographies (Germany, England, India) and environments (outdoors, indoors), as well as qualitative insights on the aligned representations.
arXiv commentAccepted as poster presentation at ICCV 2019
References in corpus (10)
- Very Deep Convolutional Networks for Large-Scale Image Recognition
- Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks
- Distributed Representations of Words and Phrases and their Compositionality
- Prototypical Networks for Few-shot Learning
- Unsupervised Domain Adaptation with Residual Transfer Networks
- Zero-Shot Learning by Convex Combination of Semantic Embeddings
- FCNs in the Wild: Pixel-level Adversarial and Constraint-based Adaptation
- The Natural Language Decathlon: Multitask Learning as Question Answering
- One Model To Learn Them All
- Decoupled Deep Neural Network for Semi-supervised Semantic Segmentation