World Programs for Model-Based Learning and Planning in Compositional State and Action Spaces
arXiv:1912.13007
Abstract
Some of the most important tasks take place in environments which lack cheap and perfect simulators, thus hampering the application of model-free reinforcement learning (RL). While model-based RL aims to learn a dynamics model, in a more general case the learner does not know a priori what the action space is. Here we propose a formalism where the learner induces a world program by learning a dynamics model and the actions in graph-based compositional environments by observing state-state transition examples. Then, the learner can perform RL with the world program as the simulator for complex planning tasks. We highlight a recent application, and propose a challenge for the community to assess world program-based planning.
Accepted at the Generative Modeling and Model-Based Reasoning for Robotics and AI workshop at ICML 2019. Presented on June 14th 2019. See https://sites.google.com/view/mbrl-icml2019
References in corpus (10)
- Learning to Plan Chemical Syntheses
- Recurrent World Models Facilitate Policy Evolution
- Challenges of Real-World Reinforcement Learning
- Transfer from Simulation to Real World through Learning Deep Inverse Dynamics Model
- Programming with a Differentiable Forth Interpreter
- Generative Code Modeling with Graphs
- Learning to Represent Edits
- Differentiable Programs with Neural Libraries
- Towards "AlphaChem": Chemical Synthesis Planning with Tree Search and Deep Neural Network Policies
- Generic Strategies for Chemical Space Exploration