Weak-shot Semantic Segmentation by Transferring Semantic Affinity and Boundary

Abstract

Weakly-supervised semantic segmentation (WSSS) with image-level labels has been widely studied to relieve the annotation burden of the traditional segmentation task. In this paper, we show that existing fully-annotated base categories can help segment objects of novel categories with only image-level labels, even if base categories and novel categories have no overlap. We refer to this task as weak-shot semantic segmentation, which could also be treated as WSSS with auxiliary fully-annotated categories. Recent advanced WSSS methods usually obtain class activation maps (CAMs) and refine them by affinity propagation. Based on the observation that semantic affinity and boundary are class-agnostic, we propose a method under the WSSS framework to transfer semantic affinity and boundary from base to novel categories. As a result, we find that pixel-level annotation of base categories can facilitate affinity learning and propagation, leading to higher-quality CAMs of novel categories. Extensive experiments on PASCAL VOC 2012 dataset prove that our method significantly outperforms WSSS baselines on novel categories.

29 pages, 8 figures

References in corpus (11)

• Learning Deconvolution Network for Semantic Segmentation
• Adversarial Learning for Semi-Supervised Semantic Segmentation
• Few-Shot Segmentation Propagation with Guided Networks
• Weakly Supervised Learning of Instance Segmentation with Inter-pixel Relations
• Self-supervised Equivariant Attention Mechanism for Weakly Supervised Semantic Segmentation
• Semi and Weakly Supervised Semantic Segmentation Using Generative Adversarial Network
• Revisiting Dilated Convolution: A Simple Approach for Weakly- and Semi- Supervised Semantic Segmentation
• Weakly-Supervised Semantic Segmentation via Sub-category Exploration
• Boosting Weakly Supervised Object Detection with Progressive Knowledge Transfer
• Cross-Supervised Object Detection
• Mixed Supervised Object Detection by Transferring Mask Prior and Semantic Similarity

Cited by in corpus (2)

• Lesion Guided Explainable Few Weak-shot Medical Report Generation
• Weak Novel Categories without Tears: A Survey on Weak-Shot Learning