Causal World Models by Unsupervised Deconfounding of Physical Dynamics

Abstract

The capability of imagining internally with a mental model of the world is vitally important for human cognition. If a machine intelligent agent can learn a world model to create a "dream" environment, it can then internally ask what-if questions -- simulate the alternative futures that haven't been experienced in the past yet -- and make optimal decisions accordingly. Existing world models are established typically by learning spatio-temporal regularities embedded from the past sensory signal without taking into account confounding factors that influence state transition dynamics. As such, they fail to answer the critical counterfactual questions about "what would have happened" if a certain action policy was taken. In this paper, we propose Causal World Models (CWMs) that allow unsupervised modeling of relationships between the intervened observations and the alternative futures by learning an estimator of the latent confounding factors. We empirically evaluate our method and demonstrate its effectiveness in a variety of physical reasoning environments. Specifically, we show reductions in sample complexity for reinforcement learning tasks and improvements in counterfactual physical reasoning.

References in corpus (27)

• Adam: A Method for Stochastic Optimization
• Learning Phrase Representations using RNN Encoder-Decoder for Statistical Machine Translation
• Empirical Evaluation of Rectified Activations in Convolutional Network
• Recurrent World Models Facilitate Policy Evolution
• Continuous Deep Q-Learning with Model-based Acceleration
• Distinguishing cause from effect using observational data: methods and benchmarks
• Doubly Robust Policy Evaluation and Learning
• Causal Effect Inference with Deep Latent-Variable Models
• Graph networks as learnable physics engines for inference and control
• Attend, Infer, Repeat: Fast Scene Understanding with Generative Models
• MONet: Unsupervised Scene Decomposition and Representation
• A Compositional Object-Based Approach to Learning Physical Dynamics
• Towards Deep Symbolic Reinforcement Learning
• The Neuro-Symbolic Concept Learner: Interpreting Scenes, Words, and Sentences From Natural Supervision
• GENESIS: Generative Scene Inference and Sampling with Object-Centric Latent Representations
• Relational Neural Expectation Maximization: Unsupervised Discovery of Objects and their Interactions
• Counterfactual Risk Minimization: Learning from Logged Bandit Feedback
• Causality for Machine Learning
• Opponent Modeling in Deep Reinforcement Learning
• Unsupervised Learning of Object Keypoints for Perception and Control
• Learning Physical Intuition of Block Towers by Example
• Counterfactuals and Policy Analysis in Structural Models
• COBRA: Data-Efficient Model-Based RL through Unsupervised Object Discovery and Curiosity-Driven Exploration
• Policy Evaluation and Optimization with Continuous Treatments
• Deconfounding Reinforcement Learning in Observational Settings
• Unsupervised Discovery of Parts, Structure, and Dynamics
• Object-Oriented Dynamics Predictor