Built-in Foreground/Background Prior for Weakly-Supervised Semantic Segmentation

Abstract

Pixel-level annotations are expensive and time consuming to obtain. Hence, weak supervision using only image tags could have a significant impact in semantic segmentation. Recently, CNN-based methods have proposed to fine-tune pre-trained networks using image tags. Without additional information, this leads to poor localization accuracy. This problem, however, was alleviated by making use of objectness priors to generate foreground/background masks. Unfortunately these priors either require training pixel-level annotations/bounding boxes, or still yield inaccurate object boundaries. Here, we propose a novel method to extract markedly more accurate masks from the pre-trained network itself, forgoing external objectness modules. This is accomplished using the activations of the higher-level convolutional layers, smoothed by a dense CRF. We demonstrate that our method, based on these masks and a weakly-supervised loss, outperforms the state-of-the-art tag-based weakly-supervised semantic segmentation techniques. Furthermore, we introduce a new form of inexpensive weak supervision yielding an additional accuracy boost.

Accepted in ECCV 2016

References in corpus (11)

• Very Deep Convolutional Networks for Large-Scale Image Recognition
• Semantic Image Segmentation with Deep Convolutional Nets and Fully Connected CRFs
• Efficient Inference in Fully Connected CRFs with Gaussian Edge Potentials
• Object Detectors Emerge in Deep Scene CNNs
• STC: A Simple to Complex Framework for Weakly-supervised Semantic Segmentation
• Learning Deconvolution Network for Semantic Segmentation
• Fully Convolutional Multi-Class Multiple Instance Learning
• BoxSup: Exploiting Bounding Boxes to Supervise Convolutional Networks for Semantic Segmentation
• DeepBox: Learning Objectness with Convolutional Networks
• DeepProposal: Hunting Objects by Cascading Deep Convolutional Layers
• DeepEdge: A Multi-Scale Bifurcated Deep Network for Top-Down Contour Detection

Cited by in corpus (15)

• Tell Me Where to Look: Guided Attention Inference Network
• Object Region Mining with Adversarial Erasing: A Simple Classification to Semantic Segmentation Approach
• Revisiting Dilated Convolution: A Simple Approach for Weakly- and Semi- Supervised Semantic Segmentation
• Incorporating Network Built-in Priors in Weakly-supervised Semantic Segmentation
• Two-Phase Learning for Weakly Supervised Object Localization
• Weakly Supervised Instance Segmentation using Class Peak Response
• Weakly-Supervised Semantic Segmentation by Iteratively Mining Common Object Features
• Exploiting saliency for object segmentation from image level labels
• CEREALS - Cost-Effective REgion-based Active Learning for Semantic Segmentation
• No More Discrimination: Cross City Adaptation of Road Scene Segmenters
• Label-PEnet: Sequential Label Propagation and Enhancement Networks for Weakly Supervised Instance Segmentation
• Self-Supervised Difference Detection for Weakly-Supervised Semantic Segmentation
• Distantly Supervised Road Segmentation
• LID 2020: The Learning from Imperfect Data Challenge Results
• Learning to segment with image-level supervision