DIPN: Deep Interaction Prediction Network with Application to Clutter Removal
arXiv:2011.04692
Abstract
We propose a Deep Interaction Prediction Network (DIPN) for learning to predict complex interactions that ensue as a robot end-effector pushes multiple objects, whose physical properties, including size, shape, mass, and friction coefficients may be unknown a priori. DIPN "imagines" the effect of a push action and generates an accurate synthetic image of the predicted outcome. DIPN is shown to be sample efficient when trained in simulation or with a real robotic system. The high accuracy of DIPN allows direct integration with a grasp network, yielding a robotic manipulation system capable of executing challenging clutter removal tasks while being trained in a fully self-supervised manner. The overall network demonstrates intelligent behavior in selecting proper actions between push and grasp for completing clutter removal tasks and significantly outperforms the previous state-of-the-art. Remarkably, DIPN achieves even better performance on the real robotic hardware system than in simulation. Videos, code, and experiments log are available at https://github.com/rutgers-arc-lab/dipn.
arXiv commentICRA 2021
References in corpus (6)
- QT-Opt: Scalable Deep Reinforcement Learning for Vision-Based Robotic Manipulation
- Feature Pyramid Networks for Object Detection
- Dex-Net 2.0: Deep Learning to Plan Robust Grasps with Synthetic Point Clouds and Analytic Grasp Metrics
- Deep Predictive Policy Training using Reinforcement Learning
- Rubik Tables and Object Rearrangement
- A Probabilistic Model for Planar Sliding of Objects with Unknown Material Properties: Identification and Robust Planning