CausalWorld: A Robotic Manipulation Benchmark for Causal Structure and Transfer Learning

Abstract

Despite recent successes of reinforcement learning (RL), it remains a challenge for agents to transfer learned skills to related environments. To facilitate research addressing this problem, we propose CausalWorld, a benchmark for causal structure and transfer learning in a robotic manipulation environment. The environment is a simulation of an open-source robotic platform, hence offering the possibility of sim-to-real transfer. Tasks consist of constructing 3D shapes from a given set of blocks - inspired by how children learn to build complex structures. The key strength of CausalWorld is that it provides a combinatorial family of such tasks with common causal structure and underlying factors (including, e.g., robot and object masses, colors, sizes). The user (or the agent) may intervene on all causal variables, which allows for fine-grained control over how similar different tasks (or task distributions) are. One can thus easily define training and evaluation distributions of a desired difficulty level, targeting a specific form of generalization (e.g., only changes in appearance or object mass). Further, this common parametrization facilitates defining curricula by interpolating between an initial and a target task. While users may define their own task distributions, we present eight meaningful distributions as concrete benchmarks, ranging from simple to very challenging, all of which require long-horizon planning as well as precise low-level motor control. Finally, we provide baseline results for a subset of these tasks on distinct training curricula and corresponding evaluation protocols, verifying the feasibility of the tasks in this benchmark.

The first two authors contributed equally, the last two authors avised jointly

References in corpus (23)

• Playing Atari with Deep Reinforcement Learning
• Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actor
• Addressing Function Approximation Error in Actor-Critic Methods
• Dota 2 with Large Scale Deep Reinforcement Learning
• Solving Rubik's Cube with a Robot Hand
• DeepMind Control Suite
• Robust Adversarial Reinforcement Learning
• Emergent Tool Use From Multi-Agent Autocurricula
• Meta-World: A Benchmark and Evaluation for Multi-Task and Meta Reinforcement Learning
• Quantifying Generalization in Reinforcement Learning
• Causality for Machine Learning
• Assessing Generalization in Deep Reinforcement Learning
• Recurrent Independent Mechanisms
• BabyAI: A Platform to Study the Sample Efficiency of Grounded Language Learning
• An empirical investigation of the challenges of real-world reinforcement learning
• InfoBot: Transfer and Exploration via the Information Bottleneck
• Reasoning About Physical Interactions with Object-Oriented Prediction and Planning
• Preparing for the Unknown: Learning a Universal Policy with Online System Identification
• Measuring the Reliability of Reinforcement Learning Algorithms
• CoPhy: Counterfactual Learning of Physical Dynamics
• Behaviour Suite for Reinforcement Learning
• TriFinger: An Open-Source Robot for Learning Dexterity
• Benchmark Environments for Multitask Learning in Continuous Domains

Cited by in corpus (11)

• A Survey of Zero-shot Generalisation in Deep Reinforcement Learning
• What Matters in Learning from Offline Human Demonstrations for Robot Manipulation
• Transferring Dexterous Manipulation from GPU Simulation to a Remote Real-World TriFinger
• Continual World: A Robotic Benchmark For Continual Reinforcement Learning
• GeneDisco: A Benchmark for Experimental Design in Drug Discovery
• Causal Curiosity: RL Agents Discovering Self-supervised Experiments for Causal Representation Learning
• On the Complexity of SHAP-Score-Based Explanations: Tractability via Knowledge Compilation and Non-Approximability Results
• CausalCity: Complex Simulations with Agency for Causal Discovery and Reasoning
• Learning Provably Robust Motion Planners Using Funnel Libraries
• Hi-Phy: A Benchmark for Hierarchical Physical Reasoning
• GalilAI: Out-of-Task Distribution Detection using Causal Active Experimentation for Safe Transfer RL