Sim-to-Real Transfer of Robot Learning with Variable Length Inputs

Abstract

Current end-to-end deep Reinforcement Learning (RL) approaches require jointly learning perception, decision-making and low-level control from very sparse reward signals and high-dimensional inputs, with little capability of incorporating prior knowledge. This results in prohibitively long training times for use on real-world robotic tasks. Existing algorithms capable of extracting task-level representations from high-dimensional inputs, e.g. object detection, often produce outputs of varying lengths, restricting their use in RL methods due to the need for neural networks to have fixed length inputs. In this work, we propose a framework that combines deep sets encoding, which allows for variable-length abstract representations, with modular RL that utilizes these representations, decoupling high-level decision making from low-level control. We successfully demonstrate our approach on the robot manipulation task of object sorting, showing that this method can learn effective policies within mere minutes of highly simplified simulation. The learned policies can be directly deployed on a robot without further training, and generalize to variations of the task unseen during training.

References in corpus (16)

• Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks
• Continuous control with deep reinforcement learning
• YOLOv3: An Incremental Improvement
• Trust Region Policy Optimization
• QT-Opt: Scalable Deep Reinforcement Learning for Vision-Based Robotic Manipulation
• IMPALA: Scalable Distributed Deep-RL with Importance Weighted Actor-Learner Architectures
• Hindsight Experience Replay
• Dex-Net 2.0: Deep Learning to Plan Robust Grasps with Synthetic Point Clouds and Analytic Grasp Metrics
• Learning Dexterous In-Hand Manipulation
• Transferring End-to-End Visuomotor Control from Simulation to Real World for a Multi-Stage Task
• Sim-to-Real: Learning Agile Locomotion For Quadruped Robots
• Investigating Human Priors for Playing Video Games
• Modular Deep Q Networks for Sim-to-real Transfer of Visuo-motor Policies
• Learning Deployable Navigation Policies at Kilometer Scale from a Single Traversal
• Cartman: The low-cost Cartesian Manipulator that won the Amazon Robotics Challenge
• Learning to Play General Video-Games via an Object Embedding Network