Incorporating Network Built-in Priors in Weakly-supervised Semantic Segmentation

Abstract

Pixel-level annotations are expensive and time consuming to obtain. Hence, weak supervision using only image tags could have a significant impact in semantic segmentation. Recently, CNN-based methods have proposed to fine-tune pre-trained networks using image tags. Without additional information, this leads to poor localization accuracy. This problem, however, was alleviated by making use of objectness priors to generate foreground/background masks. Unfortunately these priors either require pixel-level annotations/bounding boxes, or still yield inaccurate object boundaries. Here, we propose a novel method to extract accurate masks from networks pre-trained for the task of object recognition, thus forgoing external objectness modules. We first show how foreground/background masks can be obtained from the activations of higher-level convolutional layers of a network. We then show how to obtain multi-class masks by the fusion of foreground/background ones with information extracted from a weakly-supervised localization network. Our experiments evidence that exploiting these masks in conjunction with a weakly-supervised training loss yields state-of-the-art tag-based weakly-supervised semantic segmentation results.

14 pages, 11 figures, 8 tables, Accepted in IEEE Transaction on Pattern Analysis and Machine Intelligence (IEEE TPAMI)

References in corpus (14)

• Very Deep Convolutional Networks for Large-Scale Image Recognition
• Caffe: Convolutional Architecture for Fast Feature Embedding
• Semantic Image Segmentation with Deep Convolutional Nets and Fully Connected CRFs
• Efficient Inference in Fully Connected CRFs with Gaussian Edge Potentials
• Fully Convolutional Networks for Semantic Segmentation
• Conditional Random Fields as Recurrent Neural Networks
• Object Detectors Emerge in Deep Scene CNNs
• STC: A Simple to Complex Framework for Weakly-supervised Semantic Segmentation
• Fully Convolutional Multi-Class Multiple Instance Learning
• Simple Does It: Weakly Supervised Instance and Semantic Segmentation
• Built-in Foreground/Background Prior for Weakly-Supervised Semantic Segmentation
• Deep Hierarchical Parsing for Semantic Segmentation
• DeepEdge: A Multi-Scale Bifurcated Deep Network for Top-Down Contour Detection
• Weakly-Supervised Semantic Segmentation using Motion Cues

Cited by in corpus (9)

• Restricted Deformable Convolution based Road Scene Semantic Segmentation Using Surround View Cameras
• Comprehensive Attention Self-Distillation for Weakly-Supervised Object Detection
• Revisiting Dilated Convolution: A Simple Approach for Weakly- and Semi- Supervised Semantic Segmentation
• Fully Using Classifiers for Weakly Supervised Semantic Segmentation with Modified Cues
• Weakly-Supervised Learning for Tool Localization in Laparoscopic Videos
• Generating Self-Guided Dense Annotations for Weakly Supervised Semantic Segmentation
• Weakly supervised segment annotation via expectation kernel density estimation
• Weakly-supervised fire segmentation by visualizing intermediate CNN layers
• Deep Sequence Learning for Video Anticipation: From Discrete and Deterministic to Continuous and Stochastic