Schema Networks: Zero-shot Transfer with a Generative Causal Model of Intuitive Physics
arXiv:1706.04317
Abstract
The recent adaptation of deep neural network-based methods to reinforcement learning and planning domains has yielded remarkable progress on individual tasks. Nonetheless, progress on task-to-task transfer remains limited. In pursuit of efficient and robust generalization, we introduce the Schema Network, an object-oriented generative physics simulator capable of disentangling multiple causes of events and reasoning backward through causes to achieve goals. The richly structured architecture of the Schema Network can learn the dynamics of an environment directly from data. We compare Schema Networks with Asynchronous Advantage Actor-Critic and Progressive Networks on a suite of Breakout variations, reporting results on training efficiency and zero-shot generalization, consistently demonstrating faster, more robust learning and better transfer. We argue that generalizing from limited data and learning causal relationships are essential abilities on the path toward generally intelligent systems.
References in corpus (2)
Cited by in corpus (13)
- A Brief Survey of Deep Reinforcement Learning
- Deep Learning: A Critical Appraisal
- The Next Decade in AI: Four Steps Towards Robust Artificial Intelligence
- Dark, Beyond Deep: A Paradigm Shift to Cognitive AI with Humanlike Common Sense
- Human-Level Reinforcement Learning through Theory-Based Modeling, Exploration, and Planning
- Generalized Hindsight for Reinforcement Learning
- Task-Agnostic Dynamics Priors for Deep Reinforcement Learning
- Modularization of End-to-End Learning: Case Study in Arcade Games
- Structure Mapping for Transferability of Causal Models
- Computational principles of intelligence: learning and reasoning with neural networks
- Theoretically Principled Deep RL Acceleration via Nearest Neighbor Function Approximation
- Plug and Play, Model-Based Reinforcement Learning
- Episodic Memory for Learning Subjective-Timescale Models