Weakly-Supervised Semantic Segmentation by Iteratively Mining Common Object Features

Abstract

Weakly-supervised semantic segmentation under image tags supervision is a challenging task as it directly associates high-level semantic to low-level appearance. To bridge this gap, in this paper, we propose an iterative bottom-up and top-down framework which alternatively expands object regions and optimizes segmentation network. We start from initial localization produced by classification networks. While classification networks are only responsive to small and coarse discriminative object regions, we argue that, these regions contain significant common features about objects. So in the bottom-up step, we mine common object features from the initial localization and expand object regions with the mined features. To supplement non-discriminative regions, saliency maps are then considered under Bayesian framework to refine the object regions. Then in the top-down step, the refined object regions are used as supervision to train the segmentation network and to predict object masks. These object masks provide more accurate localization and contain more regions of object. Further, we take these object masks as initial localization and mine common object features from them. These processes are conducted iteratively to progressively produce fine object masks and optimize segmentation networks. Experimental results on Pascal VOC 2012 dataset demonstrate that the proposed method outperforms previous state-of-the-art methods by a large margin.

arXiv commentaccepted by CVPR 2018

References in corpus (12)

• Semantic Image Segmentation with Deep Convolutional Nets and Fully Connected CRFs
• Efficient Inference in Fully Connected CRFs with Gaussian Edge Potentials
• The Cityscapes Dataset for Semantic Urban Scene Understanding
• STC: A Simple to Complex Framework for Weakly-supervised Semantic Segmentation
• Learning Deconvolution Network for Semantic Segmentation
• Weakly- and Semi-Supervised Learning of a DCNN for Semantic Image Segmentation
• Fully Convolutional Multi-Class Multiple Instance Learning
• Learning Deep Features for Discriminative Localization
• BoxSup: Exploiting Bounding Boxes to Supervise Convolutional Networks for Semantic Segmentation
• Edge Preserving and Multi-Scale Contextual Neural Network for Salient Object Detection
• Feedforward semantic segmentation with zoom-out features
• Built-in Foreground/Background Prior for Weakly-Supervised Semantic Segmentation

Cited by in corpus (8)

• CANet: Class-Agnostic Segmentation Networks with Iterative Refinement and Attentive Few-Shot Learning
• Identity-Guided Human Semantic Parsing for Person Re-Identification
• Weakly Supervised Semantic Point Cloud Segmentation:Towards 10X Fewer Labels
• Weakly-supervised Object Localization for Few-shot Learning and Fine-grained Few-shot Learning
• Learn to Segment Retinal Lesions and Beyond
• Improve CAM with Auto-adapted Segmentation and Co-supervised Augmentation
• What Synthesis is Missing: Depth Adaptation Integrated with Weak Supervision for Indoor Scene Parsing
• Weakly Supervised Instance Segmentation by Deep Community Learning