Graph Convolutional Memory using Topological Priors
arXiv:2106.14117
Abstract
Solving partially-observable Markov decision processes (POMDPs) is critical when applying reinforcement learning to real-world problems, where agents have an incomplete view of the world. We present graph convolutional memory (GCM), the first hybrid memory model for solving POMDPs using reinforcement learning. GCM uses either human-defined or data-driven topological priors to form graph neighborhoods, combining them into a larger network topology using dynamic programming. We query the graph using graph convolution, coalescing relevant memories into a context-dependent belief. When used without human priors, GCM performs similarly to state-of-the-art methods. When used with human priors, GCM outperforms these methods on control, memorization, and navigation tasks while using significantly fewer parameters.
References in corpus (14)
- PyTorch: An Imperative Style, High-Performance Deep Learning Library
- Auto-Encoding Variational Bayes
- Empirical Evaluation of Gated Recurrent Neural Networks on Sequence Modeling
- Neural Message Passing for Quantum Chemistry
- Categorical Reparameterization with Gumbel-Softmax
- Fast Graph Representation Learning with PyTorch Geometric
- Learning to Navigate in Complex Environments
- Learning to Explore using Active Neural SLAM
- Memory-based control with recurrent neural networks
- Gated Graph Recurrent Neural Networks
- Control of Memory, Active Perception, and Action in Minecraft
- Unsupervised Predictive Memory in a Goal-Directed Agent
- Embodied Visual Navigation with Automatic Curriculum Learning in Real Environments
- The Actor Search Tree Critic (ASTC) for Off-Policy POMDP Learning in Medical Decision Making