Transferring End-to-End Visuomotor Control from Simulation to Real World for a Multi-Stage Task

Abstract

End-to-end control for robot manipulation and grasping is emerging as an attractive alternative to traditional pipelined approaches. However, end-to-end methods tend to either be slow to train, exhibit little or no generalisability, or lack the ability to accomplish long-horizon or multi-stage tasks. In this paper, we show how two simple techniques can lead to end-to-end (image to velocity) execution of a multi-stage task, which is analogous to a simple tidying routine, without having seen a single real image. This involves locating, reaching for, and grasping a cube, then locating a basket and dropping the cube inside. To achieve this, robot trajectories are computed in a simulator, to collect a series of control velocities which accomplish the task. Then, a CNN is trained to map observed images to velocities, using domain randomisation to enable generalisation to real world images. Results show that we are able to successfully accomplish the task in the real world with the ability to generalise to novel environments, including those with dynamic lighting conditions, distractor objects, and moving objects, including the basket itself. We believe our approach to be simple, highly scalable, and capable of learning long-horizon tasks that have until now not been shown with the state-of-the-art in end-to-end robot control.

arXiv comment1st Conference on Robot Learning (CoRL 2017), Mountain View, United States

References in corpus (13)

• One-Shot Imitation Learning
• Towards Vision-Based Deep Reinforcement Learning for Robotic Motion Control
• Domain Randomization for Transferring Deep Neural Networks from Simulation to the Real World
• Transfer from Simulation to Real World through Learning Deep Inverse Dynamics Model
• Third-Person Imitation Learning
• DARLA: Improving Zero-Shot Transfer in Reinforcement Learning
• Data-efficient Deep Reinforcement Learning for Dexterous Manipulation
• Learning Invariant Feature Spaces to Transfer Skills with Reinforcement Learning
• Feature Control as Intrinsic Motivation for Hierarchical Reinforcement Learning
• 3D Simulation for Robot Arm Control with Deep Q-Learning
• Multi-Modal Imitation Learning from Unstructured Demonstrations using Generative Adversarial Nets
• Deep Learning a Grasp Function for Grasping under Gripper Pose Uncertainty
• Reset-Free Guided Policy Search: Efficient Deep Reinforcement Learning with Stochastic Initial States

Cited by in corpus (44)

• How to Train Your Robot with Deep Reinforcement Learning; Lessons We've Learned
• Deep Drone Racing: From Simulation to Reality with Domain Randomization
• Reinforcement and Imitation Learning for Diverse Visuomotor Skills
• Sim-to-Real Reinforcement Learning for Deformable Object Manipulation
• Scaling data-driven robotics with reward sketching and batch reinforcement learning
• Applying Domain Randomization to Synthetic Data for Object Category Detection
• FutureMapping: The Computational Structure of Spatial AI Systems
• Sim-to-Real Transfer of Accurate Grasping with Eye-In-Hand Observations and Continuous Control
• gym-gazebo2, a toolkit for reinforcement learning using ROS 2 and Gazebo
• Learning Latent Plans from Play
• Self-driving scale car trained by Deep reinforcement learning
• Robot eye-hand coordination learning by watching human demonstrations: a task function approximation approach
• Dexterous Manipulation with Deep Reinforcement Learning: Efficient, General, and Low-Cost
• Hierarchical Foresight: Self-Supervised Learning of Long-Horizon Tasks via Visual Subgoal Generation
• An Explicit Local and Global Representation Disentanglement Framework with Applications in Deep Clustering and Unsupervised Object Detection
• RL-CycleGAN: Reinforcement Learning Aware Simulation-To-Real
• Time Reversal as Self-Supervision
• Unbiasing Semantic Segmentation For Robot Perception using Synthetic Data Feature Transfer
• Adversarial Discriminative Sim-to-real Transfer of Visuo-motor Policies
• Multi-Task Domain Adaptation for Deep Learning of Instance Grasping from Simulation
• NViSII: A Scriptable Tool for Photorealistic Image Generation
• Zero-Shot Reinforcement Learning with Deep Attention Convolutional Neural Networks
• DREAM Architecture: a Developmental Approach to Open-Ended Learning in Robotics
• End-to-end Active Object Tracking and Its Real-world Deployment via Reinforcement Learning
• Balance Between Efficient and Effective Learning: Dense2Sparse Reward Shaping for Robot Manipulation with Environment Uncertainty
• 3DCFS: Fast and Robust Joint 3D Semantic-Instance Segmentation via Coupled Feature Selection
• DESK: A Robotic Activity Dataset for Dexterous Surgical Skills Transfer to Medical Robots
• Inner Attention Supported Adaptive Cooperation for Heterogeneous Multi Robots Teaming based on Multi-agent Reinforcement Learning
• Pay attention! - Robustifying a Deep Visuomotor Policy through Task-Focused Attention
• C-3PO: Cyclic-Three-Phase Optimization for Human-Robot Motion Retargeting based on Reinforcement Learning
• Data-Efficient Learning for Complex and Real-Time Physical Problem Solving using Augmented Simulation
• Not Only Domain Randomization: Universal Policy with Embedding System Identification
• Sim-to-Real Optimization of Complex Real World Mobile Network with Imperfect Information via Deep Reinforcement Learning from Self-play
• Adversarial Feature Training for Generalizable Robotic Visuomotor Control
• Configuration Path Control
• DROID: Minimizing the Reality Gap using Single-Shot Human Demonstration
• End-to-End Egospheric Spatial Memory
• PAMTRI: Pose-Aware Multi-Task Learning for Vehicle Re-Identification Using Highly Randomized Synthetic Data
• Domain Randomization for Scene-Specific Car Detection and Pose Estimation
• DeepClaw: A Robotic Hardware Benchmarking Platform for Learning Object Manipulation
• Vision-Based Autonomous Drone Control using Supervised Learning in Simulation
• Sim-to-Real Transfer of Robot Learning with Variable Length Inputs
• Nonprehensile Riemannian Motion Predictive Control
• Adversarial Domain Randomization