One-Shot Instance Segmentation

Abstract

We tackle the problem of one-shot instance segmentation: Given an example image of a novel, previously unknown object category, find and segment all objects of this category within a complex scene. To address this challenging new task, we propose Siamese Mask R-CNN. It extends Mask R-CNN by a Siamese backbone encoding both reference image and scene, allowing it to target detection and segmentation towards the reference category. We demonstrate empirical results on MS Coco highlighting challenges of the one-shot setting: while transferring knowledge about instance segmentation to novel object categories works very well, targeting the detection network towards the reference category appears to be more difficult. Our work provides a first strong baseline for one-shot instance segmentation and will hopefully inspire further research into more powerful and flexible scene analysis algorithms. Code is available at: https://github.com/bethgelab/siamese-mask-rcnn

References in corpus (31)

• Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks
• Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networks
• YOLOv3: An Incremental Improvement
• Prototypical Networks for Few-shot Learning
• Focal Loss for Dense Object Detection
• Meta-SGD: Learning to Learn Quickly for Few-Shot Learning
• Learning to Segment Object Candidates
• Convolutional Feature Masking for Joint Object and Stuff Segmentation
• Weakly- and Semi-Supervised Learning of a DCNN for Semantic Image Segmentation
• Path Aggregation Network for Instance Segmentation
• Delta-encoder: an effective sample synthesis method for few-shot object recognition
• The 2018 DAVIS Challenge on Video Object Segmentation
• Semantic Instance Segmentation via Deep Metric Learning
• Few-Example Object Detection with Model Communication
• BoxSup: Exploiting Bounding Boxes to Supervise Convolutional Networks for Semantic Segmentation
• Zero-Shot Detection
• The 2019 DAVIS Challenge on VOS: Unsupervised Multi-Object Segmentation
• Few-Shot Segmentation Propagation with Guided Networks
• Few-Shot Learning Through an Information Retrieval Lens
• Annotating Object Instances with a Polygon-RNN
• Weakly Supervised Instance Segmentation using Class Peak Response
• RepMet: Representative-based metric learning for classification and one-shot object detection
• Large Kernel Matters -- Improve Semantic Segmentation by Global Convolutional Network
• MegDet: A Large Mini-Batch Object Detector
• Large Margin Few-Shot Learning
• Metalearning with Hebbian Fast Weights
• Comparison Network for One-Shot Conditional Object Detection
• Few-shot Object Detection via Feature Reweighting
• End-to-end Learning of Deep Visual Representations for Image Retrieval
• One-shot Texture Segmentation
• Annotation-Free and One-Shot Learning for Instance Segmentation of Homogeneous Object Clusters

Cited by in corpus (6)

• Instance-level Few-shot Learning with Class Hierarchy Mining
• FGN: Fully Guided Network for Few-Shot Instance Segmentation
• CAT: Cross-Attention Transformer for One-Shot Object Detection
• Contour Transformer Network for One-shot Segmentation of Anatomical Structures
• Localizing Infinity-shaped fishes: Sketch-guided object localization in the wild
• Learning to Segment Anatomical Structures Accurately from One Exemplar