Learning to Manipulate Deformable Objects without Demonstrations

Abstract

In this paper we tackle the problem of deformable object manipulation through model-free visual reinforcement learning (RL). In order to circumvent the sample inefficiency of RL, we propose two key ideas that accelerate learning. First, we propose an iterative pick-place action space that encodes the conditional relationship between picking and placing on deformable objects. The explicit structural encoding enables faster learning under complex object dynamics. Second, instead of jointly learning both the pick and the place locations, we only explicitly learn the placing policy conditioned on random pick points. Then, by selecting the pick point that has Maximal Value under Placing (MVP), we obtain our picking policy. This provides us with an informed picking policy during testing, while using only random pick points during training. Experimentally, this learning framework obtains an order of magnitude faster learning compared to independent action-spaces on our suite of deformable object manipulation tasks with visual RGB observations. Finally, using domain randomization, we transfer our policies to a real PR2 robot for challenging cloth and rope coverage tasks, and demonstrate significant improvements over standard RL techniques on average coverage.

Project website: https://sites.google.com/view/alternating-pick-and-place

References in corpus (16)

• Continuous control with deep reinforcement learning
• Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actor
• Trust Region Policy Optimization
• Addressing Function Approximation Error in Actor-Critic Methods
• Conditional Image Generation with PixelCNN Decoders
• Soft Actor-Critic Algorithms and Applications
• Asynchronous Methods for Deep Reinforcement Learning
• DeepMind Control Suite
• Learning Dexterous In-Hand Manipulation
• Model-Ensemble Trust-Region Policy Optimization
• Sim-to-Real Reinforcement Learning for Deformable Object Manipulation
• Robot Learning in Homes: Improving Generalization and Reducing Dataset Bias
• rlpyt: A Research Code Base for Deep Reinforcement Learning in PyTorch
• Tight open knots
• Learning Robotic Manipulation through Visual Planning and Acting
• Learning-based Feedback Controller for Deformable Object Manipulation

Cited by in corpus (7)

• Robust Policies via Mid-Level Visual Representations: An Experimental Study in Manipulation and Navigation
• Learning Arbitrary-Goal Fabric Folding with One Hour of Real Robot Experience
• Orientation Attentive Robotic Grasp Synthesis with Augmented Grasp Map Representation
• Plan2Vec: Unsupervised Representation Learning by Latent Plans
• Learning Cross-Domain Correspondence for Control with Dynamics Cycle-Consistency
• The Importance and the Limitations of Sim2Real for Robotic Manipulation in Precision Agriculture
• Learning to Manipulate Amorphous Materials