Value Propagation Networks
arXiv:1805.11199
Abstract
We present Value Propagation (VProp), a set of parameter-efficient differentiable planning modules built on Value Iteration which can successfully be trained using reinforcement learning to solve unseen tasks, has the capability to generalize to larger map sizes, and can learn to navigate in dynamic environments. We show that the modules enable learning to plan when the environment also includes stochastic elements, providing a cost-efficient learning system to build low-level size-invariant planners for a variety of interactive navigation problems. We evaluate on static and dynamic configurations of MazeBase grid-worlds, with randomly generated environments of several different sizes, and on a StarCraft navigation scenario, with more complex dynamics, and pixels as input.
arXiv commentUpdated to match ICLR 2019 OpenReview's version
References in corpus (14)
- Stabilising Experience Replay for Deep Multi-Agent Reinforcement Learning
- Sample Efficient Actor-Critic with Experience Replay
- The Predictron: End-To-End Learning and Planning
- Neural SLAM: Learning to Explore with External Memory
- Episodic Exploration for Deep Deterministic Policies: An Application to StarCraft Micromanagement Tasks
- TorchCraft: a Library for Machine Learning Research on Real-Time Strategy Games
- DESIRE: Distant Future Prediction in Dynamic Scenes with Interacting Agents
- Playing Doom with SLAM-Augmented Deep Reinforcement Learning
- MazeBase: A Sandbox for Learning from Games
- Value Prediction Network
- Memory Augmented Control Networks
- Counterfactual Reasoning about Intent for Interactive Navigation in Dynamic Environments
- Unifying task specification in reinforcement learning
- Cooperative Motion Planning for Non-Holonomic Agents with Value Iteration Networks