A Comprehensive Analysis of Weakly-Supervised Semantic Segmentation in Different Image Domains

Abstract

Recently proposed methods for weakly-supervised semantic segmentation have achieved impressive performance in predicting pixel classes despite being trained with only image labels which lack positional information. Because image annotations are cheaper and quicker to generate, weak supervision is more practical than full supervision for training segmentation algorithms. These methods have been predominantly developed to solve the background separation and partial segmentation problems presented by natural scene images and it is unclear whether they can be simply transferred to other domains with different characteristics, such as histopathology and satellite images, and still perform well. This paper evaluates state-of-the-art weakly-supervised semantic segmentation methods on natural scene, histopathology, and satellite image datasets and analyzes how to determine which method is most suitable for a given dataset. Our experiments indicate that histopathology and satellite images present a different set of problems for weakly-supervised semantic segmentation than natural scene images, such as ambiguous boundaries and class co-occurrence. Methods perform well for datasets they were developed on, but tend to perform poorly on other datasets. We present some practical techniques for these methods on unseen datasets and argue that more work is needed for a generalizable approach to weakly-supervised semantic segmentation. Our full code implementation is available on GitHub: https://github.com/lyndonchan/wsss-analysis.

arXiv comment23 pages; accepted by International Journal of Computer Vision (IJCV). Associated code available at https://github.com/lyndonchan/wsss-analysis. To view Supplementary Materials, please download pdf file listed under "Ancillary files". Int J Comput Vis (2020)

References in corpus (8)

• Efficient Inference in Fully Connected CRFs with Gaussian Edge Potentials
• DeepGlobe 2018: A Challenge to Parse the Earth through Satellite Images
• BACH: Grand Challenge on Breast Cancer Histology Images
• The ApolloScape Open Dataset for Autonomous Driving and its Application
• Assessment of algorithms for mitosis detection in breast cancer histopathology images
• Constrained-CNN losses for weakly supervised segmentation
• Fully Convolutional Multi-Class Multiple Instance Learning
• Constrained Deep Weak Supervision for Histopathology Image Segmentation

Cited by in corpus (7)

• A Survey on Deep Learning-based Architectures for Semantic Segmentation on 2D images
• Semantic Segmentation of Anaemic RBCs Using Multilevel Deep Convolutional Encoder-Decoder Network
• A drone-based framework for coral habitat mapping via weakly supervised segmentation
• Learning to segment prostate cancer by aggressiveness from scribbles in bi-parametric MRI
• Enhancing Weakly Supervised Semantic Segmentation for Fibrosis via Controllable Image Generation
• Learning with Weak Annotations for Robust Maritime Obstacle Detection
• A deep multiple instance learning approach based on coarse labels for high-resolution land-cover mapping