Probabilistic 3D Multilabel Real-time Mapping for Multi-object Manipulation

Abstract

Probabilistic 3D map has been applied to object segmentation with multiple camera viewpoints, however, conventional methods lack of real-time efficiency and functionality of multilabel object mapping. In this paper, we propose a method to generate three-dimensional map with multilabel occupancy in real-time. Extending our previous work in which only target label occupancy is mapped, we achieve multilabel object segmentation in a single looking around action. We evaluate our method by testing segmentation accuracy with 39 different objects, and applying it to a manipulation task of multiple objects in the experiments. Our mapping-based method outperforms the conventional projection-based method by 40 - 96\% relative (12.6 mean $I{U}_{3d}$), and robot successfully recognizes (86.9\%) and manipulates multiple objects (60.7\%) in an environment with heavy occlusions.

8 pages, 8 figures, IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) 2017

Cited by in corpus (2)

• Instance Segmentation of Visible and Occluded Regions for Finding and Picking Target from a Pile of Objects
• Joint Learning of Instance and Semantic Segmentation for Robotic Pick-and-Place with Heavy Occlusions in Clutter