Instance Segmentation with Point Supervision

Abstract

Instance segmentation methods often require costly per-pixel labels. We propose a method that only requires point-level annotations. During training, the model only has access to a single pixel label per object, yet the task is to output full segmentation masks. To address this challenge, we construct a network with two branches: (1) a localization network (L-Net) that predicts the location of each object; and (2) an embedding network (E-Net) that learns an embedding space where pixels of the same object are close. The segmentation masks for the located objects are obtained by grouping pixels with similar embeddings. At training time, while L-Net only requires point-level annotations, E-Net uses pseudo-labels generated by a class-agnostic object proposal method. We evaluate our approach on PASCAL VOC, COCO, KITTI and CityScapes datasets. The experiments show that our method (1) obtains competitive results compared to fully-supervised methods in certain scenarios; (2) outperforms fully- and weakly- supervised methods with a fixed annotation budget; and (3) is a first strong baseline for instance segmentation with point-level supervision.

References in corpus (9)

• Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks
• YOLOv3: An Incremental Improvement
• Learning to Segment Object Candidates
• Associative Embedding: End-to-End Learning for Joint Detection and Grouping
• Semantic Instance Segmentation via Deep Metric Learning
• Convolutional Oriented Boundaries
• RetinaMask: Learning to predict masks improves state-of-the-art single-shot detection for free
• Proposal-free Network for Instance-level Object Segmentation
• Object Counting and Instance Segmentation with Image-level Supervision

Cited by in corpus (5)

• Where are the Masks: Instance Segmentation with Image-level Supervision
• A Weakly Supervised Region-Based Active Learning Method for COVID-19 Segmentation in CT Images
• Affinity LCFCN: Learning to Segment Fish with Weak Supervision
• Weakly-supervised Instance Segmentation via Class-agnostic Learning with Salient Images
• Points2Polygons: Context-Based Segmentation from Weak Labels Using Adversarial Networks