SoftGym: Benchmarking Deep Reinforcement Learning for Deformable Object Manipulation

Abstract

Manipulating deformable objects has long been a challenge in robotics due to its high dimensional state representation and complex dynamics. Recent success in deep reinforcement learning provides a promising direction for learning to manipulate deformable objects with data driven methods. However, existing reinforcement learning benchmarks only cover tasks with direct state observability and simple low-dimensional dynamics or with relatively simple image-based environments, such as those with rigid objects. In this paper, we present SoftGym, a set of open-source simulated benchmarks for manipulating deformable objects, with a standard OpenAI Gym API and a Python interface for creating new environments. Our benchmark will enable reproducible research in this important area. Further, we evaluate a variety of algorithms on these tasks and highlight challenges for reinforcement learning algorithms, including dealing with a state representation that has a high intrinsic dimensionality and is partially observable. The experiments and analysis indicate the strengths and limitations of existing methods in the context of deformable object manipulation that can help point the way forward for future methods development. Code and videos of the learned policies can be found on our project website.

Conference on Robot Learning, 2020

References in corpus (11)

• Adam: A Method for Stochastic Optimization
• Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actor
• Learning agile and dynamic motor skills for legged robots
• DeepMind Control Suite
• Learning Latent Dynamics for Planning from Pixels
• Visual Foresight: Model-Based Deep Reinforcement Learning for Vision-Based Robotic Control
• Reinforcement Learning with Augmented Data
• Image Augmentation Is All You Need: Regularizing Deep Reinforcement Learning from Pixels
• Behaviour Suite for Reinforcement Learning
• Self-supervised Learning of Image Embedding for Continuous Control
• Reinforcement Learning without Ground-Truth State

Cited by in corpus (14)

• Orbit: A Unified Simulation Framework for Interactive Robot Learning Environments
• Industrial Internet of Things Intelligence Empowering Smart Manufacturing: A Literature Review
• Global Model Learning for Large Deformation Control of Elastic Deformable Linear Objects: An Efficient and Adaptive Approach
• Autonomous Navigation for Robot-assisted Intraluminal and Endovascular Procedures: A Systematic Review
• FlingBot: The Unreasonable Effectiveness of Dynamic Manipulation for Cloth Unfolding
• Challenges for Monocular 6D Object Pose Estimation in Robotics
• Foldsformer: Learning Sequential Multi-Step Cloth Manipulation With Space-Time Attention
• Learning Visible Connectivity Dynamics for Cloth Smoothing
• Planar Robot Casting with Real2Sim2Real Self-Supervised Learning
• Learning to Rearrange Deformable Cables, Fabrics, and Bags with Goal-Conditioned Transporter Networks
• ShakingBot: Dynamic Manipulation for Bagging
• VisuoSpatial Foresight for Physical Sequential Fabric Manipulation
• Machine Learning for Robotic Manipulation
• Robots of the Lost Arc: Self-Supervised Learning to Dynamically Manipulate Fixed-Endpoint Cables