Self-Supervised Object-in-Gripper Segmentation from Robotic Motions
arXiv:2002.04487
Abstract
Accurate object segmentation is a crucial task in the context of robotic manipulation. However, creating sufficient annotated training data for neural networks is particularly time consuming and often requires manual labeling. To this end, we propose a simple, yet robust solution for learning to segment unknown objects grasped by a robot. Specifically, we exploit motion and temporal cues in RGB video sequences. Using optical flow estimation we first learn to predict segmentation masks of our given manipulator. Then, these annotations are used in combination with motion cues to automatically distinguish between background, manipulator and unknown, grasped object. In contrast to existing systems our approach is fully self-supervised and independent of precise camera calibration, 3D models or potentially imperfect depth data. We perform a thorough comparison with alternative baselines and approaches from literature. The object masks and views are shown to be suitable training data for segmentation networks that generalize to novel environments and also allow for watertight 3D reconstruction.
15 pages, 11 figures. Video: https://www.youtube.com/watch?v=srEwuuIIgzI
References in corpus (9)
- Deep Visual Domain Adaptation: A Survey
- FlowNet: Learning Optical Flow with Convolutional Networks
- Measurement of areas on a sphere using Fibonacci and latitude-longitude lattices
- Domain Randomization for Transferring Deep Neural Networks from Simulation to the Real World
- FusionSeg: Learning to combine motion and appearance for fully automatic segmention of generic objects in videos
- Synscapes: A Photorealistic Synthetic Dataset for Street Scene Parsing
- PoseCNN: A Convolutional Neural Network for 6D Object Pose Estimation in Cluttered Scenes
- Fast Online Object Tracking and Segmentation: A Unifying Approach
- Self-Supervised Surgical Tool Segmentation using Kinematic Information