DeepMind Control Suite

Abstract

The DeepMind Control Suite is a set of continuous control tasks with a standardised structure and interpretable rewards, intended to serve as performance benchmarks for reinforcement learning agents. The tasks are written in Python and powered by the MuJoCo physics engine, making them easy to use and modify. We include benchmarks for several learning algorithms. The Control Suite is publicly available at https://www.github.com/deepmind/dm_control . A video summary of all tasks is available at http://youtu.be/rAai4QzcYbs .

24 pages, 7 figures, 2 tables

References in corpus (5)

• Adam: A Method for Stochastic Optimization
• Continuous control with deep reinforcement learning
• Prioritized Experience Replay
• Benchmarking Deep Reinforcement Learning for Continuous Control
• Reproducibility of Benchmarked Deep Reinforcement Learning Tasks for Continuous Control

Cited by in corpus (235)

• Unity: A General Platform for Intelligent Agents
• AMP: Adversarial Motion Priors for Stylized Physics-Based Character Control
• Meta-World: A Benchmark and Evaluation for Multi-Task and Meta Reinforcement Learning
• A Survey on Offline Reinforcement Learning: Taxonomy, Review, and Open Problems
• State Representation Learning for Control: An Overview
• Graph networks as learnable physics engines for inference and control
• Out-of-Distribution Generalization via Risk Extrapolation (REx)
• Challenges of Real-World Reinforcement Learning
• Reinforcement Learning with Augmented Data
• Benchmarking Model-Based Reinforcement Learning
• dm_control: Software and Tasks for Continuous Control
• A Survey of Zero-shot Generalisation in Deep Reinforcement Learning
• A Deeper Look at Experience Replay
• Latent ODEs for Irregularly-Sampled Time Series
• Dream to Control: Learning Behaviors by Latent Imagination
• Image Augmentation Is All You Need: Regularizing Deep Reinforcement Learning from Pixels
• Sim-to-Real: Learning Agile Locomotion For Quadruped Robots
• Unsupervised Learning of Object Keypoints for Perception and Control
• Critic Regularized Regression
• Learning Invariant Representations for Reinforcement Learning without Reconstruction
• Hierarchical visuomotor control of humanoids
• SoftGym: Benchmarking Deep Reinforcement Learning for Deformable Object Manipulation
• Mastering Visual Continuous Control: Improved Data-Augmented Reinforcement Learning
• Decoupling Representation Learning from Reinforcement Learning
• Stochastic Latent Actor-Critic: Deep Reinforcement Learning with a Latent Variable Model
• Self-Supervised Policy Adaptation during Deployment
• Planning to Explore via Self-Supervised World Models
• Meta reinforcement learning as task inference
• COMBO: Conservative Offline Model-Based Policy Optimization
• Automatic Data Augmentation for Generalization in Deep Reinforcement Learning
• An empirical investigation of the challenges of real-world reinforcement learning
• Data-Efficient Reinforcement Learning with Self-Predictive Representations
• Keep Doing What Worked: Behavioral Modelling Priors for Offline Reinforcement Learning
• Reinforcement Learning with Combinatorial Actions: An Application to Vehicle Routing
• SUNRISE: A Simple Unified Framework for Ensemble Learning in Deep Reinforcement Learning
• Rigorous Agent Evaluation: An Adversarial Approach to Uncover Catastrophic Failures
• Bootstrap Latent-Predictive Representations for Multitask Reinforcement Learning
• MinAtar: An Atari-Inspired Testbed for Thorough and Reproducible Reinforcement Learning Experiments
• Reinforcement Learning with Prototypical Representations
• Relative Entropy Regularized Policy Iteration
• Robust Reinforcement Learning for Continuous Control with Model Misspecification
• Visual Spatial Attention and Proprioceptive Data-Driven Reinforcement Learning for Robust Peg-in-Hole Task Under Variable Conditions
• Invariant Causal Prediction for Block MDPs
• Stabilizing Deep Q-Learning with ConvNets and Vision Transformers under Data Augmentation
• Energy Shaping Control of a CyberOctopus Soft Arm
• MushroomRL: Simplifying Reinforcement Learning Research
• Investigating Generalisation in Continuous Deep Reinforcement Learning
• Neural Graph Evolution: Towards Efficient Automatic Robot Design
• Deterministic Implementations for Reproducibility in Deep Reinforcement Learning
• Behaviour Suite for Reinforcement Learning
• DeepRacer: Educational Autonomous Racing Platform for Experimentation with Sim2Real Reinforcement Learning
• A Self-Tuning Actor-Critic Algorithm
• CausalWorld: A Robotic Manipulation Benchmark for Causal Structure and Transfer Learning
• Interactive Differentiable Simulation
• Q-Learning in enormous action spaces via amortized approximate maximization
• Value constrained model-free continuous control
• Rewriting History with Inverse RL: Hindsight Inference for Policy Improvement
• Continuous-Discrete Reinforcement Learning for Hybrid Control in Robotics
• Online and Offline Reinforcement Learning by Planning with a Learned Model
• Gym-Ignition: Reproducible Robotic Simulations for Reinforcement Learning
• NeoRL: A Near Real-World Benchmark for Offline Reinforcement Learning
• Learning Functionally Decomposed Hierarchies for Continuous Control Tasks with Path Planning
• TriFinger: An Open-Source Robot for Learning Dexterity
• Evolutionary reinforcement learning of dynamical large deviations
• Predictive Information Accelerates Learning in RL
• Knowledge Transfer for Cross-Domain Reinforcement Learning: A Systematic Review
• Sample-efficient Reinforcement Learning Representation Learning with Curiosity Contrastive Forward Dynamics Model
• TF-Replicator: Distributed Machine Learning for Researchers
• DoorGym: A Scalable Door Opening Environment And Baseline Agent
• Brax -- A Differentiable Physics Engine for Large Scale Rigid Body Simulation
• ARCHER: Aggressive Rewards to Counter bias in Hindsight Experience Replay
• Exploration-Enhanced POLITEX
• Model-Based Offline Planning
• Modeling Continuous Stochastic Processes with Dynamic Normalizing Flows
• Scheduled Intrinsic Drive: A Hierarchical Take on Intrinsically Motivated Exploration
• Learning to Manipulate Deformable Objects without Demonstrations
• GPU-Accelerated Robotic Simulation for Distributed Reinforcement Learning
• Dynamics-aware Embeddings
• Integrating Contrastive Learning with Dynamic Models for Reinforcement Learning from Images
• Debiased Contrastive Learning
• Unsupervised Learning of Object Structure and Dynamics from Videos
• Accelerating Reinforcement Learning through GPU Atari Emulation
• Generalization in Reinforcement Learning by Soft Data Augmentation
• Harnessing Structures for Value-Based Planning and Reinforcement Learning
• Texar: A Modularized, Versatile, and Extensible Toolkit for Text Generation
• On the role of planning in model-based deep reinforcement learning
• Hamilton-Jacobi Deep Q-Learning for Deterministic Continuous-Time Systems with Lipschitz Continuous Controls
• Learning Visual Robotic Control Efficiently with Contrastive Pre-training and Data Augmentation
• Behavior Priors for Efficient Reinforcement Learning
• SECANT: Self-Expert Cloning for Zero-Shot Generalization of Visual Policies
• RLAD: Reinforcement Learning from Pixels for Autonomous Driving in Urban Environments
• Scalable Coordinated Exploration in Concurrent Reinforcement Learning
• Using a thousand optimization tasks to learn hyperparameter search strategies
• Learning Task-relevant Representations for Generalization via Characteristic Functions of Reward Sequence Distributions
• The Distracting Control Suite -- A Challenging Benchmark for Reinforcement Learning from Pixels
• Observation Space Matters: Benchmark and Optimization Algorithm
• Tactical Optimism and Pessimism for Deep Reinforcement Learning
• Learning Actionable Representations from Visual Observations
• Constraint Learning for Control Tasks with Limited Duration Barrier Functions
• Multimodal Information Bottleneck for Deep Reinforcement Learning with Multiple Sensors
• Model-Predictive Control via Cross-Entropy and Gradient-Based Optimization
• Logarithmic regret for episodic continuous-time linear-quadratic reinforcement learning over a finite-time horizon
• Robust Predictable Control
• Self-Imitation Learning via Generalized Lower Bound Q-learning
• CompILE: Compositional Imitation Learning and Execution
• ROBEL: Robotics Benchmarks for Learning with Low-Cost Robots
• MALI: A memory efficient and reverse accurate integrator for Neural ODEs
• Positive-Unlabeled Reward Learning
• Motion Perception in Reinforcement Learning with Dynamic Objects
• Deep neuroethology of a virtual rodent
• Learning and Planning in Complex Action Spaces
• Actor-Critic Reinforcement Learning for Control with Stability Guarantee
• Self-Supervised Sim-to-Real Adaptation for Visual Robotic Manipulation
• Dreaming: Model-based Reinforcement Learning by Latent Imagination without Reconstruction
• Automating Reinforcement Learning with Example-based Resets
• Improving Robustness via Risk Averse Distributional Reinforcement Learning
• CORA: Benchmarks, Baselines, and Metrics as a Platform for Continual Reinforcement Learning Agents
• AI-MOLE: Autonomous Iterative Motion Learning for Unknown Nonlinear Dynamics with Extensive Experimental Validation
• PlaNet of the Bayesians: Reconsidering and Improving Deep Planning Network by Incorporating Bayesian Inference
• Maximum Mutation Reinforcement Learning for Scalable Control
• Efficient Model-Based Reinforcement Learning through Optimistic Policy Search and Planning
• The Benchmark Lottery
• Arena: A General Evaluation Platform and Building Toolkit for Multi-Agent Intelligence
• Fever Basketball: A Complex, Flexible, and Asynchronized Sports Game Environment for Multi-agent Reinforcement Learning
• PlayVirtual: Augmenting Cycle-Consistent Virtual Trajectories for Reinforcement Learning
• How to Make Deep RL Work in Practice
• Learning a Multi-Modal Policy via Imitating Demonstrations with Mixed Behaviors
• Tonic: A Deep Reinforcement Learning Library for Fast Prototyping and Benchmarking
• Mutual Information Maximization for Robust Plannable Representations
• Local Search for Policy Iteration in Continuous Control
• rl_reach: Reproducible Reinforcement Learning Experiments for Robotic Reaching Tasks
• Contrastive Variational Reinforcement Learning for Complex Observations
• The Differentiable Cross-Entropy Method
• Exploratory State Representation Learning
• Acceleration of Actor-Critic Deep Reinforcement Learning for Visual Grasping in Clutter by State Representation Learning Based on Disentanglement of a Raw Input Image
• Forward Prediction for Physical Reasoning
• Sample-based Distributional Policy Gradient
• Physically Embedded Planning Problems: New Challenges for Reinforcement Learning
• Balancing Constraints and Rewards with Meta-Gradient D4PG
• Colored Noise in PPO: Improved Exploration and Performance through Correlated Action Sampling
• Improving Computational Efficiency in Visual Reinforcement Learning via Stored Embeddings
• Using Unity to Help Solve Intelligence
• Scalable Modular Synthetic Data Generation for Advancing Aerial Autonomy
• Bridging Imagination and Reality for Model-Based Deep Reinforcement Learning
• Online Hyper-parameter Tuning in Off-policy Learning via Evolutionary Strategies
• Attention-Privileged Reinforcement Learning
• Domain Adversarial Reinforcement Learning
• Information Theoretic Model Predictive Q-Learning
• Deluca -- A Differentiable Control Library: Environments, Methods, and Benchmarking
• Blue River Controls: A toolkit for Reinforcement Learning Control Systems on Hardware
• Task-Agnostic Morphology Evolution
• TRAIL: Near-Optimal Imitation Learning with Suboptimal Data
• Efficient Deep Reinforcement Learning via Adaptive Policy Transfer
• Learning Robust State Abstractions for Hidden-Parameter Block MDPs
• Advances in Preference-based Reinforcement Learning: A Review
• Learning Cross-Domain Correspondence for Control with Dynamics Cycle-Consistency
• Model Primitive Hierarchical Lifelong Reinforcement Learning
• Robust Constrained Reinforcement Learning for Continuous Control with Model Misspecification
• Continuous-Time Model-Based Reinforcement Learning
• Models, Pixels, and Rewards: Evaluating Design Trade-offs in Visual Model-Based Reinforcement Learning
• Quinoa: a Q-function You Infer Normalized Over Actions
• Unsupervised Visual Attention and Invariance for Reinforcement Learning
• Auto-Tuned Sim-to-Real Transfer
• Learning Dynamics Models for Model Predictive Agents
• Decoupling Value and Policy for Generalization in Reinforcement Learning
• A learning gap between neuroscience and reinforcement learning
• DRIBO: Robust Deep Reinforcement Learning via Multi-View Information Bottleneck
• Block Contextual MDPs for Continual Learning
• Cross-Domain Imitation Learning via Optimal Transport
• Domain-Adversarial and Conditional State Space Model for Imitation Learning
• Towards Automatic Actor-Critic Solutions to Continuous Control
• Agent Environment Cycle Games
• On the Search for Feedback in Reinforcement Learning
• Reinforced Imitation Learning by Free Energy Principle
• An Actor-Critic-Attention Mechanism for Deep Reinforcement Learning in Multi-view Environments
• PBCS : Efficient Exploration and Exploitation Using a Synergy between Reinforcement Learning and Motion Planning
• Arena: a toolkit for Multi-Agent Reinforcement Learning
• Scenic4RL: Programmatic Modeling and Generation of Reinforcement Learning Environments
• Approximate Model-Based Shielding for Safe Reinforcement Learning
• Variance Reduction for Evolution Strategies via Structured Control Variates
• Nested Mixture of Experts: Cooperative and Competitive Learning of Hybrid Dynamical System
• Pre-training of Deep RL Agents for Improved Learning under Domain Randomization
• Curiosity creates Diversity in Policy Search
• Unlocking Pixels for Reinforcement Learning via Implicit Attention
• CLOUD: Contrastive Learning of Unsupervised Dynamics
• Pretrained Encoders are All You Need
• Distributed Uplink Beamforming in Cell-Free Networks Using Deep Reinforcement Learning
• Making Curiosity Explicit in Vision-based RL
• Learning Temporally-Consistent Representations for Data-Efficient Reinforcement Learning
• Analytic Manifold Learning: Unifying and Evaluating Representations for Continuous Control
• Reinforcement Learning with Latent Flow
• Optimization Issues in KL-Constrained Approximate Policy Iteration
• Improved Regret Bound and Experience Replay in Regularized Policy Iteration
• Temporally-Extended ε-Greedy Exploration
• Baconian: A Unified Open-source Framework for Model-Based Reinforcement Learning
• ACE: An Actor Ensemble Algorithm for Continuous Control with Tree Search
• Continual Reinforcement Learning with Diversity Exploration and Adversarial Self-Correction
• SQUIRL: Robust and Efficient Learning from Video Demonstration of Long-Horizon Robotic Manipulation Tasks
• Towards Robust Bisimulation Metric Learning
• Weak Human Preference Supervision For Deep Reinforcement Learning
• Continuous-time Value Function Approximation in Reproducing Kernel Hilbert Spaces
• Reinforcement Learning in the Wild with Maximum Likelihood-based Model Transfer
• Data-Driven Reinforcement Learning for Virtual Character Animation Control
• Learning to Plan Optimistically: Uncertainty-Guided Deep Exploration via Latent Model Ensembles
• Worrisome Properties of Neural Network Controllers and Their Symbolic Representations
• Cubic Spline Smoothing Compensation for Irregularly Sampled Sequences
• Sufficiently Accurate Model Learning
• Momentum-based Accelerated Q-learning
• Learning Representations for Pixel-based Control: What Matters and Why?
• IMPACT: Importance Weighted Asynchronous Architectures with Clipped Target Networks
• Disentangled (Un)Controllable Features
• Benchmarking the Spectrum of Agent Capabilities
• Measuring Progress in Deep Reinforcement Learning Sample Efficiency
• Learning Calibratable Policies using Programmatic Style-Consistency
• Learn Proportional Derivative Controllable Latent Space from Pixels
• Accelerated Target Updates for Q-learning
• Human-like Planning for Reaching in Cluttered Environments
• Reactive Environments for Active Inference Agents with RxEnvironments.jl
• A Decoupled Data Based Approach to Stochastic Optimal Control Problems
• End-to-end grasping policies for human-in-the-loop robots via deep reinforcement learning
• Sufficiently Accurate Model Learning for Planning
• Learning in Sparse Rewards settings through Quality-Diversity algorithms
• Continuous Control for High-Dimensional State Spaces: An Interactive Learning Approach
• Utilizing Skipped Frames in Action Repeats via Pseudo-Actions
• Learning to Reach, Swim, Walk and Fly in One Trial: Data-Driven Control with Scarce Data and Side Information
• GEM: Group Enhanced Model for Learning Dynamical Control Systems
• Koopman Spectrum Nonlinear Regulators and Efficient Online Learning
• Few-Shot Image-to-Semantics Translation for Policy Transfer in Reinforcement Learning
• A Geometric Perspective on Self-Supervised Policy Adaptation
• Diagnosing and exploiting the computational demands of videos games for deep reinforcement learning
• State Representation Learning from Demonstration
• Contrastive Active Inference
• Decoupled Exploration and Exploitation Policies for Sample-Efficient Reinforcement Learning
• Deep Reinforcement Learning for Tactile Robotics: Learning to Type on a Braille Keyboard
• Discovering a set of policies for the worst case reward