Resolving Spurious Correlations in Causal Models of Environments via Interventions
arXiv:2002.05217
Abstract
Causal models bring many benefits to decision-making systems (or agents) by making them interpretable, sample-efficient, and robust to changes in the input distribution. However, spurious correlations can lead to wrong causal models and predictions. We consider the problem of inferring a causal model of a reinforcement learning environment and we propose a method to deal with spurious correlations. Specifically, our method designs a reward function that incentivizes an agent to do an intervention to find errors in the causal model. The data obtained from doing the intervention is used to improve the causal model. We propose several intervention design methods and compare them. The experimental results in a grid-world environment show that our approach leads to better causal models compared to baselines: learning the model on data from a random policy or a policy trained on the environment's reward. The main contribution consists of methods to design interventions to resolve spurious correlations.
arXiv comment9 pages, 7 figures, 3 pages supplementary material
References in corpus (15)
- Playing Atari with Deep Reinforcement Learning
- Dota 2 with Large Scale Deep Reinforcement Learning
- Solving Rubik's Cube with a Robot Hand
- Data-Efficient Hierarchical Reinforcement Learning
- Causal Confusion in Imitation Learning
- The Consciousness Prior
- A Philosophical Treatise of Universal Induction
- Adversarial Policies: Attacking Deep Reinforcement Learning
- Visual Causal Feature Learning
- Disentangling the independently controllable factors of variation by interacting with the world
- Learning Neural Causal Models from Unknown Interventions
- Woulda, Coulda, Shoulda: Counterfactually-Guided Policy Search
- Thompson Sampling is Asymptotically Optimal in General Environments
- Understanding Agent Incentives using Causal Influence Diagrams. Part I: Single Action Settings
- Learning Causal Models Online