Where are the Masks: Instance Segmentation with Image-level Supervision

Abstract

A major obstacle in instance segmentation is that existing methods often need many per-pixel labels in order to be effective. These labels require large human effort and for certain applications, such labels are not readily available. To address this limitation, we propose a novel framework that can effectively train with image-level labels, which are significantly cheaper to acquire. For instance, one can do an internet search for the term "car" and obtain many images where a car is present with minimal effort. Our framework consists of two stages: (1) train a classifier to generate pseudo masks for the objects of interest; (2) train a fully supervised Mask R-CNN on these pseudo masks. Our two main contribution are proposing a pipeline that is simple to implement and is amenable to different segmentation methods; and achieves new state-of-the-art results for this problem setup. Our results are based on evaluating our method on PASCAL VOC 2012, a standard dataset for weakly supervised methods, where we demonstrate major performance gains compared to existing methods with respect to mean average precision.

Accepted at BMVC2019

References in corpus (13)

• Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks
• The Cityscapes Dataset for Semantic Urban Scene Understanding
• What makes for effective detection proposals?
• Learning to Segment Object Candidates
• Feature Pyramid Networks for Object Detection
• BoxSup: Exploiting Bounding Boxes to Supervise Convolutional Networks for Semantic Segmentation
• Convolutional Oriented Boundaries
• RetinaMask: Learning to predict masks improves state-of-the-art single-shot detection for free
• Weakly Supervised Instance Segmentation using Class Peak Response
• Multiple Instance Detection Network with Online Instance Classifier Refinement
• Object Counting and Instance Segmentation with Image-level Supervision
• Instance Segmentation with Point Supervision
• Instance Segmentation of Fibers from Low Resolution CT Scans via 3D Deep Embedding Learning

Cited by in corpus (5)

• Solve the Puzzle of Instance Segmentation in Videos: A Weakly Supervised Framework with Spatio-Temporal Collaboration
• Self-supervised Learning for Panoptic Segmentation of Multiple Fruit Flower Species
• Weakly-supervised Instance Segmentation via Class-agnostic Learning with Salient Images
• Complete Instances Mining for Weakly Supervised Instance Segmentation
• Mask-guided sample selection for Semi-Supervised Instance Segmentation