XLVIN: eXecuted Latent Value Iteration Nets
arXiv:2010.13146
Abstract
Value Iteration Networks (VINs) have emerged as a popular method to incorporate planning algorithms within deep reinforcement learning, enabling performance improvements on tasks requiring long-range reasoning and understanding of environment dynamics. This came with several limitations, however: the model is not incentivised in any way to perform meaningful planning computations, the underlying state space is assumed to be discrete, and the Markov decision process (MDP) is assumed fixed and known. We propose eXecuted Latent Value Iteration Networks (XLVINs), which combine recent developments across contrastive self-supervised learning, graph representation learning and neural algorithmic reasoning to alleviate all of the above limitations, successfully deploying VIN-style models on generic environments. XLVINs match the performance of VIN-like models when the underlying MDP is discrete, fixed and known, and provides significant improvements to model-free baselines across three general MDP setups.
arXiv commentNeurIPS 2020 Deep Reinforcement Learning Workshop
References in corpus (16)
- Statistical mechanics of complex networks
- PyTorch: An Imperative Style, High-Performance Deep Learning Library
- Neural Message Passing for Quantum Chemistry
- Geometric deep learning: going beyond Euclidean data
- Striving for Simplicity: The All Convolutional Net
- Recurrent World Models Facilitate Policy Evolution
- Learning Latent Dynamics for Planning from Pixels
- Reinforcement Learning with Unsupervised Auxiliary Tasks
- The Predictron: End-To-End Learning and Planning
- Metrics for Finite Markov Decision Processes
- Learning Dynamic Belief Graphs to Generalize on Text-Based Games
- TreeQN and ATreeC: Differentiable Tree-Structured Models for Deep Reinforcement Learning
- Generalized Value Iteration Networks: Life Beyond Lattices
- Pointer Graph Networks
- Neural Bipartite Matching
- Graph neural induction of value iteration