The Distracting Control Suite -- A Challenging Benchmark for Reinforcement Learning from Pixels

Abstract

Robots have to face challenging perceptual settings, including changes in viewpoint, lighting, and background. Current simulated reinforcement learning (RL) benchmarks such as DM Control provide visual input without such complexity, which limits the transfer of well-performing methods to the real world. In this paper, we extend DM Control with three kinds of visual distractions (variations in background, color, and camera pose) to produce a new challenging benchmark for vision-based control, and we analyze state of the art RL algorithms in these settings. Our experiments show that current RL methods for vision-based control perform poorly under distractions, and that their performance decreases with increasing distraction complexity, showing that new methods are needed to cope with the visual complexities of the real world. We also find that combinations of multiple distraction types are more difficult than a mere combination of their individual effects.

Code available at https://github.com/google-research/google-research/tree/master/distracting_control

References in corpus (9)

• Solving Rubik's Cube with a Robot Hand
• DeepMind Control Suite
• Learning Latent Dynamics for Planning from Pixels
• Reinforcement Learning with Augmented Data
• Image Augmentation Is All You Need: Regularizing Deep Reinforcement Learning from Pixels
• Learning Invariant Representations for Reinforcement Learning without Reconstruction
• Self-Supervised Policy Adaptation during Deployment
• Scaling data-driven robotics with reward sketching and batch reinforcement learning
• Natural Environment Benchmarks for Reinforcement Learning

Cited by in corpus (8)

• A Survey of Zero-shot Generalisation in Deep Reinforcement Learning
• Contrastive Behavioral Similarity Embeddings for Generalization in Reinforcement Learning
• DreamerPro: Reconstruction-Free Model-Based Reinforcement Learning with Prototypical Representations
• Unlocking Pixels for Reinforcement Learning via Implicit Attention
• Learning Representations for Pixel-based Control: What Matters and Why?
• Robust Robotic Control from Pixels using Contrastive Recurrent State-Space Models
• Contrastive Active Inference
• A Geometric Perspective on Self-Supervised Policy Adaptation