Learning RGB-D Feature Embeddings for Unseen Object Instance Segmentation

Abstract

Segmenting unseen objects in cluttered scenes is an important skill that robots need to acquire in order to perform tasks in new environments. In this work, we propose a new method for unseen object instance segmentation by learning RGB-D feature embeddings from synthetic data. A metric learning loss function is utilized to learn to produce pixel-wise feature embeddings such that pixels from the same object are close to each other and pixels from different objects are separated in the embedding space. With the learned feature embeddings, a mean shift clustering algorithm can be applied to discover and segment unseen objects. We further improve the segmentation accuracy with a new two-stage clustering algorithm. Our method demonstrates that non-photorealistic synthetic RGB and depth images can be used to learn feature embeddings that transfer well to real-world images for unseen object instance segmentation.

CoRL 2020

References in corpus (9)

• Adam: A Method for Stochastic Optimization
• Very Deep Convolutional Networks for Large-Scale Image Recognition
• A Simple Framework for Contrastive Learning of Visual Representations
• Semantic Instance Segmentation with a Discriminative Loss Function
• Dex-Net 2.0: Deep Learning to Plan Robust Grasps with Synthetic Point Clouds and Analytic Grasp Metrics
• Deep Object Pose Estimation for Semantic Robotic Grasping of Household Objects
• Dense Object Nets: Learning Dense Visual Object Descriptors By and For Robotic Manipulation
• The Best of Both Modes: Separately Leveraging RGB and Depth for Unseen Object Instance Segmentation
• Unseen Object Instance Segmentation for Robotic Environments

Cited by in corpus (5)

• RF-Annotate: Automatic RF-Supervised Image Annotation of Common Objects in Context
• SORNet: Spatial Object-Centric Representations for Sequential Manipulation
• Reinforcement Learning for Vision-based Object Manipulation with Non-parametric Policy and Action Primitives
• ICM-3D: Instantiated Category Modeling for 3D Instance Segmentation
• Unknown Object Segmentation through Domain Adaptation