Pre-training of Deep RL Agents for Improved Learning under Domain Randomization
arXiv:2104.14386
Abstract
Visual domain randomization in simulated environments is a widely used method to transfer policies trained in simulation to real robots. However, domain randomization and augmentation hamper the training of a policy. As reinforcement learning struggles with a noisy training signal, this additional nuisance can drastically impede training. For difficult tasks it can even result in complete failure to learn. To overcome this problem we propose to pre-train a perception encoder that already provides an embedding invariant to the randomization. We demonstrate that this yields consistently improved results on a randomized version of DeepMind control suite tasks and a stacking environment on arbitrary backgrounds with zero-shot transfer to a physical robot.
References in corpus (15)
- Adam: A Method for Stochastic Optimization
- A Simple Framework for Contrastive Learning of Visual Representations
- Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actor
- Solving Rubik's Cube with a Robot Hand
- DeepMind Control Suite
- Learning Latent Dynamics for Planning from Pixels
- Dex-Net 2.0: Deep Learning to Plan Robust Grasps with Synthetic Point Clouds and Analytic Grasp Metrics
- Reinforcement Learning with Augmented Data
- Driving Policy Transfer via Modularity and Abstraction
- Dream to Control: Learning Behaviors by Latent Imagination
- Image Augmentation Is All You Need: Regularizing Deep Reinforcement Learning from Pixels
- Sim-to-Real Robot Learning from Pixels with Progressive Nets
- Reinforcement and Imitation Learning for Diverse Visuomotor Skills
- Learning a visuomotor controller for real world robotic grasping using simulated depth images
- Spatial Broadcast Decoder: A Simple Architecture for Learning Disentangled Representations in VAEs