Domain Randomization for Transferring Deep Neural Networks from Simulation to the Real World

Abstract

Bridging the 'reality gap' that separates simulated robotics from experiments on hardware could accelerate robotic research through improved data availability. This paper explores domain randomization, a simple technique for training models on simulated images that transfer to real images by randomizing rendering in the simulator. With enough variability in the simulator, the real world may appear to the model as just another variation. We focus on the task of object localization, which is a stepping stone to general robotic manipulation skills. We find that it is possible to train a real-world object detector that is accurate to $1.5$cm and robust to distractors and partial occlusions using only data from a simulator with non-realistic random textures. To demonstrate the capabilities of our detectors, we show they can be used to perform grasping in a cluttered environment. To our knowledge, this is the first successful transfer of a deep neural network trained only on simulated RGB images (without pre-training on real images) to the real world for the purpose of robotic control.

8 pages, 7 figures. Submitted to 2017 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS 2017)

References in corpus (13)

• Adam: A Method for Stochastic Optimization
• Very Deep Convolutional Networks for Large-Scale Image Recognition
• Deep Domain Confusion: Maximizing for Domain Invariance
• Unsupervised Cross-Domain Image Generation
• Revisiting Batch Normalization For Practical Domain Adaptation
• Playing for Data: Ground Truth from Computer Games
• EPOpt: Learning Robust Neural Network Policies Using Model Ensembles
• Render for CNN: Viewpoint Estimation in Images Using CNNs Trained with Rendered 3D Model Views
• 3D Simulation for Robot Arm Control with Deep Q-Learning
• Preparing for the Unknown: Learning a Universal Policy with Online System Identification
• Reinforcement Learning for Pivoting Task
• How useful is photo-realistic rendering for visual learning?
• Deep Predictive Policy Training using Reinforcement Learning

Cited by in corpus (94)

• An Introduction to Deep Reinforcement Learning
• Solving Rubik's Cube with a Robot Hand
• Hindsight Experience Replay
• Learning Dexterous In-Hand Manipulation
• Learning by Playing - Solving Sparse Reward Tasks from Scratch
• Quantifying Generalization in Reinforcement Learning
• A Berkeley View of Systems Challenges for AI
• Virtual to Real Reinforcement Learning for Autonomous Driving
• Reverse Curriculum Generation for Reinforcement Learning
• Transferring End-to-End Visuomotor Control from Simulation to Real World for a Multi-Stage Task
• Scalable agent alignment via reward modeling: a research direction
• Sim-to-Real: Learning Agile Locomotion For Quadruped Robots
• Reinforcement and Imitation Learning for Diverse Visuomotor Skills
• Sim-to-Real Reinforcement Learning for Deformable Object Manipulation
• Learning to Walk in the Real World with Minimal Human Effort
• Universal Planning Networks
• HoME: a Household Multimodal Environment
• A Survey of Deep Network Solutions for Learning Control in Robotics: From Reinforcement to Imitation
• Domain Stylization: A Strong, Simple Baseline for Synthetic to Real Image Domain Adaptation
• Policy Transfer with Strategy Optimization
• Multi-Agent Common Knowledge Reinforcement Learning
• Transferring Dexterous Manipulation from GPU Simulation to a Remote Real-World TriFinger
• Network Randomization: A Simple Technique for Generalization in Deep Reinforcement Learning
• Scaling data-driven robotics with reward sketching and batch reinforcement learning
• Domain Generalization with MixStyle
• Autonomous Quadrotor Landing using Deep Reinforcement Learning
• Active Neural Localization
• Applying Domain Randomization to Synthetic Data for Object Category Detection
• Sim-to-Real Transfer of Accurate Grasping with Eye-In-Hand Observations and Continuous Control
• Counterfactual Generative Networks
• ManiSkill: Generalizable Manipulation Skill Benchmark with Large-Scale Demonstrations
• Autonomous Driving in Reality with Reinforcement Learning and Image Translation
• Towards Generalization and Simplicity in Continuous Control
• How to Train a CAT: Learning Canonical Appearance Transformations for Direct Visual Localization Under Illumination Change
• Selecting Data Augmentation for Simulating Interventions
• DoorGym: A Scalable Door Opening Environment And Baseline Agent
• Reinforcement Learning of Active Vision for Manipulating Objects under Occlusions
• Mutual Alignment Transfer Learning
• Learning of Parameters in Behavior Trees for Movement Skills
• MixStyle Neural Networks for Domain Generalization and Adaptation
• Transferring Autonomous Driving Knowledge on Simulated and Real Intersections
• 3D-OES: Viewpoint-Invariant Object-Factorized Environment Simulators
• Time Reversal as Self-Supervision
• Unbiasing Semantic Segmentation For Robot Perception using Synthetic Data Feature Transfer
• Learning Fast Adaptation with Meta Strategy Optimization
• Learning and Planning with a Semantic Model
• OptLayer - Practical Constrained Optimization for Deep Reinforcement Learning in the Real World
• Multi-Task Domain Adaptation for Deep Learning of Instance Grasping from Simulation
• Discovering Blind Spots in Reinforcement Learning
• Learning to Grasp from a Single Demonstration
• Adversarial Discriminative Sim-to-real Transfer of Visuo-motor Policies
• ORRB -- OpenAI Remote Rendering Backend
• Map-based Multi-Policy Reinforcement Learning: Enhancing Adaptability of Robots by Deep Reinforcement Learning
• NViSII: A Scriptable Tool for Photorealistic Image Generation
• ROS2Learn: a reinforcement learning framework for ROS 2
• Deep Reinforcement Learning to Acquire Navigation Skills for Wheel-Legged Robots in Complex Environments
• A New Autoregressive Neural Network Model with Command Compensation for Imitation Learning Based on Bilateral Control
• Cross-Domain Perceptual Reward Functions
• Learning from scarce information: using synthetic data to classify Roman fine ware pottery
• Can Synthetic Data Improve Object Detection Results for Remote Sensing Images?
• Trustable and Automated Machine Learning Running with Blockchain and Its Applications
• Grounding Language for Transfer in Deep Reinforcement Learning
• Seeing All the Angles: Learning Multiview Manipulation Policies for Contact-Rich Tasks from Demonstrations
• Zero in on Shape: A Generic 2D-3D Instance Similarity Metric learned from Synthetic Data
• One-Shot Unsupervised Cross-Domain Detection
• Interferobot: aligning an optical interferometer by a reinforcement learning agent
• Few Shot System Identification for Reinforcement Learning
• Self-Supervised Object-in-Gripper Segmentation from Robotic Motions
• A User's Guide to Calibrating Robotics Simulators
• Auto-Tuned Sim-to-Real Transfer
• Learning to Locomote with Deep Neural-Network and CPG-based Control in a Soft Snake Robot
• Neuro-algorithmic Policies enable Fast Combinatorial Generalization
• AutoPilot: Automating SoC Design Space Exploration for SWaP Constrained Autonomous UAVs
• A Fog Robotic System for Dynamic Visual Servoing
• MetaGrasp: Data Efficient Grasping by Affordance Interpreter Network
• Avoidance of Manual Labeling in Robotic Autonomous Navigation Through Multi-Sensory Semi-Supervised Learning
• Constrained Exploration and Recovery from Experience Shaping
• A Learning Framework for Robust Bin Picking by Customized Grippers
• A gamified simulator and physical platform for self-driving algorithm training and validation
• Introducing Pose Consistency and Warp-Alignment for Self-Supervised 6D Object Pose Estimation in Color Images
• Meta Arcade: A Configurable Environment Suite for Meta-Learning
• Sim2Air - Synthetic aerial dataset for UAV monitoring
• A general approach to bridge the reality-gap
• Learning Kinematic Descriptions using SPARE: Simulated and Physical ARticulated Extendable dataset
• Sim-to-Real Optimization of Complex Real World Mobile Network with Imperfect Information via Deep Reinforcement Learning from Self-play
• A Few Shot Adaptation of Visual Navigation Skills to New Observations using Meta-Learning
• Autonomous Industrial Management via Reinforcement Learning: Self-Learning Agents for Decision-Making -- A Review
• CrDoCo: Pixel-level Domain Transfer with Cross-Domain Consistency
• Generation of GelSight Tactile Images for Sim2Real Learning
• End-to-end grasping policies for human-in-the-loop robots via deep reinforcement learning
• Transferring Agent Behaviors from Videos via Motion GANs
• Deep Active Localization
• Adaptation of Quadruped Robot Locomotion with Meta-Learning
• Domain Randomization for Scene-Specific Car Detection and Pose Estimation