Joint Learning of Saliency Detection and Weakly Supervised Semantic Segmentation

Abstract

Existing weakly supervised semantic segmentation (WSSS) methods usually utilize the results of pre-trained saliency detection (SD) models without explicitly modeling the connections between the two tasks, which is not the most efficient configuration. Here we propose a unified multi-task learning framework to jointly solve WSSS and SD using a single network, \ie saliency, and segmentation network (SSNet). SSNet consists of a segmentation network (SN) and a saliency aggregation module (SAM). For an input image, SN generates the segmentation result and, SAM predicts the saliency of each category and aggregating the segmentation masks of all categories into a saliency map. The proposed network is trained end-to-end with image-level category labels and class-agnostic pixel-level saliency labels. Experiments on PASCAL VOC 2012 segmentation dataset and four saliency benchmark datasets show the performance of our method compares favorably against state-of-the-art weakly supervised segmentation methods and fully supervised saliency detection methods.

arXiv commentAccepted by ICCV19

References in corpus (8)

• Adam: A Method for Stochastic Optimization
• Very Deep Convolutional Networks for Large-Scale Image Recognition
• Efficient Inference in Fully Connected CRFs with Gaussian Edge Potentials
• DeepSaliency: Multi-Task Deep Neural Network Model for Salient Object Detection
• Weakly- and Semi-Supervised Learning of a DCNN for Semantic Image Segmentation
• An Unsupervised Game-Theoretic Approach to Saliency Detection
• Salient Objects in Clutter: Bringing Salient Object Detection to the Foreground
• Self-explanatory Deep Salient Object Detection

Cited by in corpus (15)

• RGB-D Salient Object Detection: A Survey
• Advances in Deep Concealed Scene Understanding
• Uncertainty-aware Joint Salient Object and Camouflaged Object Detection
• Group-Wise Semantic Mining for Weakly Supervised Semantic Segmentation
• An interpretable classifier for high-resolution breast cancer screening images utilizing weakly supervised localization
• Generalized Multi-Task Learning from Substantially Unlabeled Multi-Source Medical Image Data
• Weakly Supervised Semantic Segmentation by Pixel-to-Prototype Contrast
• Weakly-supervised Instance Segmentation via Class-agnostic Learning with Salient Images
• High-Resolution Image Inpainting with Iterative Confidence Feedback and Guided Upsampling
• BiconNet: An Edge-preserved Connectivity-based Approach for Salient Object Detection
• LID 2020: The Learning from Imperfect Data Challenge Results
• Multi-Miner: Object-Adaptive Region Mining for Weakly-Supervised Semantic Segmentation
• Gradient-Induced Co-Saliency Detection
• Weakly Supervised Instance Segmentation by Deep Community Learning
• Maximize the Exploration of Congeneric Semantics for Weakly Supervised Semantic Segmentation