Semi and Weakly Supervised Semantic Segmentation Using Generative Adversarial Network

Abstract

Semantic segmentation has been a long standing challenging task in computer vision. It aims at assigning a label to each image pixel and needs significant number of pixellevel annotated data, which is often unavailable. To address this lack, in this paper, we leverage, on one hand, massive amount of available unlabeled or weakly labeled data, and on the other hand, non-real images created through Generative Adversarial Networks. In particular, we propose a semi-supervised framework ,based on Generative Adversarial Networks (GANs), which consists of a generator network to provide extra training examples to a multi-class classifier, acting as discriminator in the GAN framework, that assigns sample a label y from the K possible classes or marks it as a fake sample (extra class). The underlying idea is that adding large fake visual data forces real samples to be close in the feature space, enabling a bottom-up clustering process, which, in turn, improves multiclass pixel classification. To ensure higher quality of generated images for GANs with consequent improved pixel classification, we extend the above framework by adding weakly annotated data, i.e., we provide class level information to the generator. We tested our approaches on several challenging benchmarking visual datasets, i.e. PASCAL, SiftFLow, Stanford and CamVid, achieving competitive performance also compared to state-of-the-art semantic segmentation method

References in corpus (11)

• Conditional Generative Adversarial Nets
• Unsupervised Representation Learning with Deep Convolutional Generative Adversarial Networks
• Fully Convolutional Networks for Semantic Segmentation
• Learning Deconvolution Network for Semantic Segmentation
• Weakly- and Semi-Supervised Learning of a DCNN for Semantic Image Segmentation
• Unsupervised and Semi-supervised Learning with Categorical Generative Adversarial Networks
• Fully Convolutional Multi-Class Multiple Instance Learning
• Fully Connected Deep Structured Networks
• How (not) to Train your Generative Model: Scheduled Sampling, Likelihood, Adversary?
• BoxSup: Exploiting Bounding Boxes to Supervise Convolutional Networks for Semantic Segmentation
• Feedforward semantic segmentation with zoom-out features

Cited by in corpus (12)

• Generalizing semi-supervised generative adversarial networks to regression using feature contrasting
• Simultaneously Localize, Segment and Rank the Camouflaged Objects
• Identity-Guided Human Semantic Parsing for Person Re-Identification
• Improving Semantic Segmentation via Self-Training
• Unsupervised Domain Adaptation in Semantic Segmentation: a Review
• Uncertainty-aware Joint Salient Object and Camouflaged Object Detection
• Deep Cross-Modal Audio-Visual Generation
• Fast video object segmentation with Spatio-Temporal GANs
• Weak-shot Semantic Segmentation by Transferring Semantic Affinity and Boundary
• Judge the Judges: A Large-Scale Evaluation Study of Neural Language Models for Online Review Generation
• Coarse-to-fine Semantic Segmentation from Image-level Labels
• Medical Image Segmentation with Limited Supervision: A Review of Deep Network Models