Sim-to-Real Robot Learning from Pixels with Progressive Nets

Abstract

Applying end-to-end learning to solve complex, interactive, pixel-driven control tasks on a robot is an unsolved problem. Deep Reinforcement Learning algorithms are too slow to achieve performance on a real robot, but their potential has been demonstrated in simulated environments. We propose using progressive networks to bridge the reality gap and transfer learned policies from simulation to the real world. The progressive net approach is a general framework that enables reuse of everything from low-level visual features to high-level policies for transfer to new tasks, enabling a compositional, yet simple, approach to building complex skills. We present an early demonstration of this approach with a number of experiments in the domain of robot manipulation that focus on bridging the reality gap. Unlike other proposed approaches, our real-world experiments demonstrate successful task learning from raw visual input on a fully actuated robot manipulator. Moreover, rather than relying on model-based trajectory optimisation, the task learning is accomplished using only deep reinforcement learning and sparse rewards.

Cited by in corpus (40)

• Solving Rubik's Cube with a Robot Hand
• Deep Drone Racing: From Simulation to Reality with Domain Randomization
• Learning Dexterous In-Hand Manipulation
• Active Learning in Robotics: A Review of Control Principles
• Assessing Transferability from Simulation to Reality for Reinforcement Learning
• Never Stop Learning: The Effectiveness of Fine-Tuning in Robotic Reinforcement Learning
• DeepRacer: Educational Autonomous Racing Platform for Experimentation with Sim2Real Reinforcement Learning
• Learning to Map Natural Language Instructions to Physical Quadcopter Control using Simulated Flight
• Analysing Deep Reinforcement Learning Agents Trained with Domain Randomisation
• Asymmetric self-play for automatic goal discovery in robotic manipulation
• VR-Goggles for Robots: Real-to-sim Domain Adaptation for Visual Control
• Deep Learning for Embodied Vision Navigation: A Survey
• Domain Adaptation Through Task Distillation
• PEBBLE: Feedback-Efficient Interactive Reinforcement Learning via Relabeling Experience and Unsupervised Pre-training
• SimGAN: Hybrid Simulator Identification for Domain Adaptation via Adversarial Reinforcement Learning
• On the Transfer of Disentangled Representations in Realistic Settings
• Diverse Auto-Curriculum is Critical for Successful Real-World Multiagent Learning Systems
• A review of mobile robot motion planning methods: from classical motion planning workflows to reinforcement learning-based architectures
• ROS2Learn: a reinforcement learning framework for ROS 2
• Zero-Shot Reinforcement Learning with Deep Attention Convolutional Neural Networks
• How to Close Sim-Real Gap? Transfer with Segmentation!
• Robust Domain Randomised Reinforcement Learning through Peer-to-Peer Distillation
• Human-Robot Collaboration via Deep Reinforcement Learning of Real-World Interactions
• Learning to Navigate from Simulation via Spatial and Semantic Information Synthesis with Noise Model Embedding
• Scalable sim-to-real transfer of soft robot designs
• Hierarchically Integrated Models: Learning to Navigate from Heterogeneous Robots
• Robot Action Selection Learning via Layered Dimension Informed Program Synthesis
• Continual Learning: Tackling Catastrophic Forgetting in Deep Neural Networks with Replay Processes
• SIM2REALVIZ: Visualizing the Sim2Real Gap in Robot Ego-Pose Estimation
• A Simple Approach to Continual Learning by Transferring Skill Parameters
• Pre-training of Deep RL Agents for Improved Learning under Domain Randomization
• DROID: Minimizing the Reality Gap using Single-Shot Human Demonstration
• Comparing Task Simplifications to Learn Closed-Loop Object Picking Using Deep Reinforcement Learning
• REGRAD: A Large-Scale Relational Grasp Dataset for Safe and Object-Specific Robotic Grasping in Clutter
• Continuous Deep Q-Learning with Simulator for Stabilization of Uncertain Discrete-Time Systems
• Policy Transfer across Visual and Dynamics Domain Gaps via Iterative Grounding
• A Bayesian Approach to Reinforcement Learning of Vision-Based Vehicular Control
• Nonprehensile Riemannian Motion Predictive Control
• Can Q-Learning be Improved with Advice?
• GrowSpace: Learning How to Shape Plants