Cloth Manipulation Planning on Basis of Mesh Representations with Incomplete Domain Knowledge and Voxel-to-Mesh Estimation

Abstract

We consider the problem of open-goal planning for robotic cloth manipulation. Core of our system is a neural network trained as a forward model of cloth behaviour under manipulation, with planning performed through backpropagation. We introduce a neural network-based routine for estimating mesh representations from voxel input, and perform planning in mesh format internally. We address the problem of planning with incomplete domain knowledge by means of an explicit epistemic uncertainty signal. This signal is calculated from prediction divergence between two instances of the forward model network and used to avoid epistemic uncertainty during planning. Finally, we introduce logic for handling restriction of grasp points to a discrete set of candidates, in order to accommodate graspability constraints imposed by robotic hardware. We evaluate the system's mesh estimation, prediction, and planning ability on simulated cloth for sequences of one to three manipulations. Comparative experiments confirm that planning on basis of estimated meshes improves accuracy compared to voxel-based planning, and that epistemic uncertainty avoidance improves performance under conditions of incomplete domain knowledge. Planning time cost is a few seconds. We additionally present qualitative results on robot hardware.

27 pages, 13 figures

References in corpus (8)

• Obfuscated Gradients Give a False Sense of Security: Circumventing Defenses to Adversarial Examples
• State Representation Learning for Control: An Overview
• Visual Foresight: Model-Based Deep Reinforcement Learning for Vision-Based Robotic Control
• Learning Predictive Representations for Deformable Objects Using Contrastive Estimation
• MPC-Inspired Neural Network Policies for Sequential Decision Making
• Dynamic Manipulation of Flexible Objects with Torque Sequence Using a Deep Neural Network
• VisuoSpatial Foresight for Multi-Step, Multi-Task Fabric Manipulation
• An Energy Minimization Approach to 3D Non-Rigid Deformable Surface Estimation Using RGBD Data