Weakly Supervised Instance Segmentation using Class Peak Response

Abstract

Weakly supervised instance segmentation with image-level labels, instead of expensive pixel-level masks, remains unexplored. In this paper, we tackle this challenging problem by exploiting class peak responses to enable a classification network for instance mask extraction. With image labels supervision only, CNN classifiers in a fully convolutional manner can produce class response maps, which specify classification confidence at each image location. We observed that local maximums, i.e., peaks, in a class response map typically correspond to strong visual cues residing inside each instance. Motivated by this, we first design a process to stimulate peaks to emerge from a class response map. The emerged peaks are then back-propagated and effectively mapped to highly informative regions of each object instance, such as instance boundaries. We refer to the above maps generated from class peak responses as Peak Response Maps (PRMs). PRMs provide a fine-detailed instance-level representation, which allows instance masks to be extracted even with some off-the-shelf methods. To the best of our knowledge, we for the first time report results for the challenging image-level supervised instance segmentation task. Extensive experiments show that our method also boosts weakly supervised pointwise localization as well as semantic segmentation performance, and reports state-of-the-art results on popular benchmarks, including PASCAL VOC 2012 and MS COCO.

arXiv commentAccepted in CVPR 2018 (Spotlight)

References in corpus (14)

• R-FCN: Object Detection via Region-based Fully Convolutional Networks
• Fully Convolutional Networks for Semantic Segmentation
• STC: A Simple to Complex Framework for Weakly-supervised Semantic Segmentation
• Multiscale Combinatorial Grouping for Image Segmentation and Object Proposal Generation
• Weakly- and Semi-Supervised Learning of a DCNN for Semantic Image Segmentation
• BoxSup: Exploiting Bounding Boxes to Supervise Convolutional Networks for Semantic Segmentation
• Instance-aware Semantic Segmentation via Multi-task Network Cascades
• Pixel-level Encoding and Depth Layering for Instance-level Semantic Labeling
• Monocular Object Instance Segmentation and Depth Ordering with CNNs
• Built-in Foreground/Background Prior for Weakly-Supervised Semantic Segmentation
• Soft Proposal Networks for Weakly Supervised Object Localization
• Instance-Level Segmentation for Autonomous Driving with Deep Densely Connected MRFs
• From Image-level to Pixel-level Labeling with Convolutional Networks
• Convolutional Oriented Boundaries: From Image Segmentation to High-Level Tasks

Cited by in corpus (14)

• A Survey on Explainable Artificial Intelligence (XAI): Towards Medical XAI
• One-Shot Instance Segmentation
• Deep Weakly-Supervised Learning Methods for Classification and Localization in Histology Images: A Survey
• Identity-Guided Human Semantic Parsing for Person Re-Identification
• Where are the Masks: Instance Segmentation with Image-level Supervision
• TextMountain: Accurate Scene Text Detection via Instance Segmentation
• Weakly-supervised Salient Instance Detection
• Class Activation Map Generation by Representative Class Selection and Multi-Layer Feature Fusion
• Mask Based Unsupervised Content Transfer
• Deep Discriminative Representation Learning with Attention Map for Scene Classification
• Learning Segmentation Masks with the Independence Prior
• Weak Novel Categories without Tears: A Survey on Weak-Shot Learning
• Weakly Supervised Instance Segmentation Using Hybrid Network
• Weakly Supervised Instance Segmentation by Deep Community Learning