Transferring End-to-End Visuomotor Control from Simulation to Real World for a Multi-Stage Task

Abstract

End-to-end control for robot manipulation and grasping is emerging as an attractive alternative to traditional pipelined approaches. However, end-to-end methods tend to either be slow to train, exhibit little or no generalisability, or lack the ability to accomplish long-horizon or multi-stage tasks. In this paper, we show how two simple techniques can lead to end-to-end (image to velocity) execution of a multi-stage task, which is analogous to a simple tidying routine, without having seen a single real image. This involves locating, reaching for, and grasping a cube, then locating a basket and dropping the cube inside. To achieve this, robot trajectories are computed in a simulator, to collect a series of control velocities which accomplish the task. Then, a CNN is trained to map observed images to velocities, using domain randomisation to enable generalisation to real world images. Results show that we are able to successfully accomplish the task in the real world with the ability to generalise to novel environments, including those with dynamic lighting conditions, distractor objects, and moving objects, including the basket itself. We believe our approach to be simple, highly scalable, and capable of learning long-horizon tasks that have until now not been shown with the state-of-the-art in end-to-end robot control.

1st Conference on Robot Learning (CoRL 2017), Mountain View, United States

References in corpus (5)

• Domain Randomization for Transferring Deep Neural Networks from Simulation to the Real World
• Learning Invariant Feature Spaces to Transfer Skills with Reinforcement Learning
• 3D Simulation for Robot Arm Control with Deep Q-Learning
• Deep Learning a Grasp Function for Grasping under Gripper Pose Uncertainty
• Reset-Free Guided Policy Search: Efficient Deep Reinforcement Learning with Stochastic Initial States

Cited by in corpus (18)

• How to Train Your Robot with Deep Reinforcement Learning; Lessons We've Learned
• Sim-to-Real Transfer of Accurate Grasping with Eye-In-Hand Observations and Continuous Control
• gym-gazebo2, a toolkit for reinforcement learning using ROS 2 and Gazebo
• An Explicit Local and Global Representation Disentanglement Framework with Applications in Deep Clustering and Unsupervised Object Detection
• RL-CycleGAN: Reinforcement Learning Aware Simulation-To-Real
• NViSII: A Scriptable Tool for Photorealistic Image Generation
• Zero-Shot Reinforcement Learning with Deep Attention Convolutional Neural Networks
• DREAM Architecture: a Developmental Approach to Open-Ended Learning in Robotics
• Balance Between Efficient and Effective Learning: Dense2Sparse Reward Shaping for Robot Manipulation with Environment Uncertainty
• 3DCFS: Fast and Robust Joint 3D Semantic-Instance Segmentation via Coupled Feature Selection
• DESK: A Robotic Activity Dataset for Dexterous Surgical Skills Transfer to Medical Robots
• Data-Efficient Learning for Complex and Real-Time Physical Problem Solving using Augmented Simulation
• Not Only Domain Randomization: Universal Policy with Embedding System Identification
• End-to-End Egospheric Spatial Memory
• DROID: Minimizing the Reality Gap using Single-Shot Human Demonstration
• DeepClaw: A Robotic Hardware Benchmarking Platform for Learning Object Manipulation
• PAMTRI: Pose-Aware Multi-Task Learning for Vehicle Re-Identification Using Highly Randomized Synthetic Data
• Vision-Based Autonomous Drone Control using Supervised Learning in Simulation