Memory Augmented Control Networks

Abstract

Planning problems in partially observable environments cannot be solved directly with convolutional networks and require some form of memory. But, even memory networks with sophisticated addressing schemes are unable to learn intelligent reasoning satisfactorily due to the complexity of simultaneously learning to access memory and plan. To mitigate these challenges we introduce the Memory Augmented Control Network (MACN). The proposed network architecture consists of three main parts. The first part uses convolutions to extract features and the second part uses a neural network-based planning module to pre-plan in the environment. The third part uses a network controller that learns to store those specific instances of past information that are necessary for planning. The performance of the network is evaluated in discrete grid world environments for path planning in the presence of simple and complex obstacles. We show that our network learns to plan and can generalize to new environments.

References in corpus (4)

• Neural SLAM: Learning to Explore with External Memory
• Neural Map: Structured Memory for Deep Reinforcement Learning
• Learning Transferable Policies for Monocular Reactive MAV Control
• Neural Network Memory Architectures for Autonomous Robot Navigation

Cited by in corpus (19)

• A Survey of Deep Network Solutions for Learning Control in Robotics: From Reinforcement to Imitation
• Unifying Map and Landmark Based Representations for Visual Navigation
• Learning to Navigate in Cities Without a Map
• Metalearned Neural Memory
• Scene Memory Transformer for Embodied Agents in Long-Horizon Tasks
• Value Propagation Networks
• Scalable Centralized Deep Multi-Agent Reinforcement Learning via Policy Gradients
• A Short Survey On Memory Based Reinforcement Learning
• Learning and Planning with a Semantic Model
• Simultaneous Mapping and Target Driven Navigation
• Inverse reinforcement learning for autonomous navigation via differentiable semantic mapping and planning
• The act of remembering: a study in partially observable reinforcement learning
• Learning Navigation Costs from Demonstration with Semantic Observations
• Motion Planning for Heterogeneous Unmanned Systems under Partial Observation from UAV
• A Novel Deep Neural Network Architecture for Mars Visual Navigation
• Deep Pepper: Expert Iteration based Chess agent in the Reinforcement Learning Setting
• Sufficiently Accurate Model Learning
• Hard Attention Control By Mutual Information Maximization
• How memory architecture affects learning in a simple POMDP: the two-hypothesis testing problem