Learning Predictive Representations for Deformable Objects Using Contrastive Estimation

Abstract

Using visual model-based learning for deformable object manipulation is challenging due to difficulties in learning plannable visual representations along with complex dynamic models. In this work, we propose a new learning framework that jointly optimizes both the visual representation model and the dynamics model using contrastive estimation. Using simulation data collected by randomly perturbing deformable objects on a table, we learn latent dynamics models for these objects in an offline fashion. Then, using the learned models, we use simple model-based planning to solve challenging deformable object manipulation tasks such as spreading ropes and cloths. Experimentally, we show substantial improvements in performance over standard model-based learning techniques across our rope and cloth manipulation suite. Finally, we transfer our visual manipulation policies trained on data purely collected in simulation to a real PR2 robot through domain randomization.

arXiv commentProject website: https://sites.google.com/view/contrastive-predictive-model

References in corpus (16)

• Distributed Representations of Words and Phrases and their Compositionality
• Continuous control with deep reinforcement learning
• A Simple Framework for Contrastive Learning of Visual Representations
• Representation Learning with Contrastive Predictive Coding
• Trust Region Policy Optimization
• Soft Actor-Critic Algorithms and Applications
• InfoGAN: Interpretable Representation Learning by Information Maximizing Generative Adversarial Nets
• Contrastive Multiview Coding
• VEEGAN: Reducing Mode Collapse in GANs using Implicit Variational Learning
• Visual Foresight: Model-Based Deep Reinforcement Learning for Vision-Based Robotic Control
• Learning to Poke by Poking: Experiential Learning of Intuitive Physics
• Learning Plannable Representations with Causal InfoGAN
• Robot Learning in Homes: Improving Generalization and Reducing Dataset Bias
• Tight open knots
• Learning Robotic Manipulation through Visual Planning and Acting
• Learning-based Feedback Controller for Deformable Object Manipulation

Cited by in corpus (27)

• Global Model Learning for Large Deformation Control of Elastic Deformable Linear Objects: An Efficient and Adaptive Approach
• Mastering Visual Continuous Control: Improved Data-Augmented Reinforcement Learning
• Decoupling Representation Learning from Reinforcement Learning
• Self-Supervised Policy Adaptation during Deployment
• Reinforcement Learning with Prototypical Representations
• Keypoints into the Future: Self-Supervised Correspondence in Model-Based Reinforcement Learning
• Foldsformer: Learning Sequential Multi-Step Cloth Manipulation With Space-Time Attention
• Robust Policies via Mid-Level Visual Representations: An Experimental Study in Manipulation and Navigation
• Learning Arbitrary-Goal Fabric Folding with One Hour of Real Robot Experience
• Learning Dense Visual Correspondences in Simulation to Smooth and Fold Real Fabrics
• Deep Reinforcement Learning Based on Local GNN for Goal-conditioned Deformable Object Rearranging
• Learning Visible Connectivity Dynamics for Cloth Smoothing
• Learning Visual Robotic Control Efficiently with Contrastive Pre-training and Data Augmentation
• Planar Robot Casting with Real2Sim2Real Self-Supervised Learning
• Learning to Rearrange Deformable Cables, Fabrics, and Bags with Goal-Conditioned Transporter Networks
• Dreaming: Model-based Reinforcement Learning by Latent Imagination without Reconstruction
• VisuoSpatial Foresight for Physical Sequential Fabric Manipulation
• Bimanual rope manipulation skill synthesis through context dependent correction policy learning from human demonstration
• Graph-Transporter: A Graph-based Learning Method for Goal-Conditioned Deformable Object Rearranging Task
• Untangling Dense Knots by Learning Task-Relevant Keypoints
• Learning Cross-Domain Correspondence for Control with Dynamics Cycle-Consistency
• DeformerNet: A Deep Learning Approach to 3D Deformable Object Manipulation
• Robots of the Lost Arc: Self-Supervised Learning to Dynamically Manipulate Fixed-Endpoint Cables
• CLOUD: Contrastive Learning of Unsupervised Dynamics
• Cloth Manipulation Planning on Basis of Mesh Representations with Incomplete Domain Knowledge and Voxel-to-Mesh Estimation
• Naive Artificial Intelligence
• Learning a Skill-sequence-dependent Policy for Long-horizon Manipulation Tasks