Relevance-Guided Modeling of Object Dynamics for Reinforcement Learning

Abstract

Current deep reinforcement learning (RL) approaches incorporate minimal prior knowledge about the environment, limiting computational and sample efficiency. \textit{Objects} provide a succinct and causal description of the world, and many recent works have proposed unsupervised object representation learning using priors and losses over static object properties like visual consistency. However, object dynamics and interactions are also critical cues for objectness. In this paper we propose a framework for reasoning about object dynamics and behavior to rapidly determine minimal and task-specific object representations. To demonstrate the need to reason over object behavior and dynamics, we introduce a suite of RGBD MuJoCo object collection and avoidance tasks that, while intuitive and visually simple, confound state-of-the-art unsupervised object representation learning algorithms. We also highlight the potential of this framework on several Atari games, using our object representation and standard RL and planning algorithms to learn dramatically faster than existing deep RL algorithms.

References in corpus (15)

• Scalable trust-region method for deep reinforcement learning using Kronecker-factored approximation
• Model-Based Reinforcement Learning for Atari
• Multi-Object Representation Learning with Iterative Variational Inference
• Hybrid Reward Architecture for Reinforcement Learning
• Schema Networks: Zero-shot Transfer with a Generative Causal Model of Intuitive Physics
• Visual Dynamics: Probabilistic Future Frame Synthesis via Cross Convolutional Networks
• Recurrent Environment Simulators
• Unsupervised Learning of Object Keypoints for Perception and Control
• SPACE: Unsupervised Object-Oriented Scene Representation via Spatial Attention and Decomposition
• Unsupervised Video Object Segmentation for Deep Reinforcement Learning
• The Best of Both Modes: Separately Leveraging RGB and Depth for Unseen Object Instance Segmentation
• Entity Abstraction in Visual Model-Based Reinforcement Learning
• Object-Oriented Dynamics Predictor
• Unsupervised Discovery of 3D Physical Objects from Video
• AlignNet: Unsupervised Entity Alignment