Object-Oriented Dynamics Predictor

Abstract

Generalization has been one of the major challenges for learning dynamics models in model-based reinforcement learning. However, previous work on action-conditioned dynamics prediction focuses on learning the pixel-level motion and thus does not generalize well to novel environments with different object layouts. In this paper, we present a novel object-oriented framework, called object-oriented dynamics predictor (OODP), which decomposes the environment into objects and predicts the dynamics of objects conditioned on both actions and object-to-object relations. It is an end-to-end neural network and can be trained in an unsupervised manner. To enable the generalization ability of dynamics learning, we design a novel CNN-based relation mechanism that is class-specific (rather than object-specific) and exploits the locality principle. Empirical results show that OODP significantly outperforms previous methods in terms of generalization over novel environments with various object layouts. OODP is able to learn from very few environments and accurately predict dynamics in a large number of unseen environments. In addition, OODP learns semantically and visually interpretable dynamics models.

arXiv commentAccepted to NIPS 2018

Cited by in corpus (11)

• Unsupervised State Representation Learning in Atari
• Deep Imitation Learning for Bimanual Robotic Manipulation
• Transformation-based Adversarial Video Prediction on Large-Scale Data
• Physics-as-Inverse-Graphics: Unsupervised Physical Parameter Estimation from Video
• Counterfactual Data Augmentation using Locally Factored Dynamics
• Uncovering Closed-form Governing Equations of Nonlinear Dynamics from Videos
• Causal World Models by Unsupervised Deconfounding of Physical Dynamics
• Object-Oriented Dynamics Learning through Multi-Level Abstraction
• Joint Perception and Control as Inference with an Object-based Implementation
• Relevance-Guided Modeling of Object Dynamics for Reinforcement Learning
• Zero-shot Policy Learning with Spatial Temporal RewardDecomposition on Contingency-aware Observation