Ab Initio Particle-based Object Manipulation
arXiv:2107.08865 · doi:10.15607/RSS.2021.XVII.071
Abstract
This paper presents Particle-based Object Manipulation (Prompt), a new approach to robot manipulation of novel objects ab initio, without prior object models or pre-training on a large object data set. The key element of Prompt is a particle-based object representation, in which each particle represents a point in the object, the local geometric, physical, and other features of the point, and also its relation with other particles. Like the model-based analytic approaches to manipulation, the particle representation enables the robot to reason about the object's geometry and dynamics in order to choose suitable manipulation actions. Like the data-driven approaches, the particle representation is learned online in real-time from visual sensor input, specifically, multi-view RGB images. The particle representation thus connects visual perception with robot control. Prompt combines the benefits of both model-based reasoning and data-driven learning. We show empirically that Prompt successfully handles a variety of everyday objects, some of which are transparent. It handles various manipulation tasks, including grasping, pushing, etc,. Our experiments also show that Prompt outperforms a state-of-the-art data-driven grasping method on the daily objects, even though it does not use any offline training data.
arXiv commentRobotics: Science and Systems (RSS) 2021
References in corpus (6)
- U-Net: Going Deeper with Nested U-Structure for Salient Object Detection
- ShapeNet: An Information-Rich 3D Model Repository
- QT-Opt: Scalable Deep Reinforcement Learning for Vision-Based Robotic Manipulation
- Benchmarking in Manipulation Research: The YCB Object and Model Set and Benchmarking Protocols
- Learning a visuomotor controller for real world robotic grasping using simulated depth images
- DiffTaichi: Differentiable Programming for Physical Simulation