Systematic Evaluation of Causal Discovery in Visual Model Based Reinforcement Learning
arXiv:2107.00848
Abstract
Inducing causal relationships from observations is a classic problem in machine learning. Most work in causality starts from the premise that the causal variables themselves are observed. However, for AI agents such as robots trying to make sense of their environment, the only observables are low-level variables like pixels in images. To generalize well, an agent must induce high-level variables, particularly those which are causal or are affected by causal variables. A central goal for AI and causality is thus the joint discovery of abstract representations and causal structure. However, we note that existing environments for studying causal induction are poorly suited for this objective because they have complicated task-specific causal graphs which are impossible to manipulate parametrically (e.g., number of nodes, sparsity, causal chain length, etc.). In this work, our goal is to facilitate research in learning representations of high-level variables as well as causal structures among them. In order to systematically probe the ability of methods to identify these variables and structures, we design a suite of benchmarking RL environments. We evaluate various representation learning algorithms from the literature and find that explicitly incorporating structure and modularity in models can help causal induction in model-based reinforcement learning.
References in corpus (22)
- Adam: A Method for Stochastic Optimization
- Neural Message Passing for Quantum Chemistry
- Interaction Networks for Learning about Objects, Relations and Physics
- Meta-World: A Benchmark and Evaluation for Multi-Task and Meta Reinforcement Learning
- DAGs with NO TEARS: Continuous Optimization for Structure Learning
- Quantifying Generalization in Reinforcement Learning
- Relational Neural Expectation Maximization: Unsupervised Discovery of Objects and their Interactions
- A Meta-Transfer Objective for Learning to Disentangle Causal Mechanisms
- Gotta Learn Fast: A New Benchmark for Generalization in RL
- Causal Reasoning from Meta-reinforcement Learning
- Recurrent Independent Mechanisms
- Contrastive Learning of Structured World Models
- CLEVRER: CoLlision Events for Video REpresentation and Reasoning
- BabyAI: A Platform to Study the Sample Efficiency of Grounded Language Learning
- COBRA: Data-Efficient Model-Based RL through Unsupervised Object Discovery and Curiosity-Driven Exploration
- Causal Induction from Visual Observations for Goal Directed Tasks
- Learning Neural Causal Models from Unknown Interventions
- CoPhy: Counterfactual Learning of Physical Dynamics
- Behaviour Suite for Reinforcement Learning
- Causally Correct Partial Models for Reinforcement Learning
- Object Files and Schemata: Factorizing Declarative and Procedural Knowledge in Dynamical Systems
- Amortized learning of neural causal representations