Neural SLAM: Learning to Explore with External Memory

Abstract

We present an approach for agents to learn representations of a global map from sensor data, to aid their exploration in new environments. To achieve this, we embed procedures mimicking that of traditional Simultaneous Localization and Mapping (SLAM) into the soft attention based addressing of external memory architectures, in which the external memory acts as an internal representation of the environment. This structure encourages the evolution of SLAM-like behaviors inside a completely differentiable deep neural network. We show that this approach can help reinforcement learning agents to successfully explore new environments where long-term memory is essential. We validate our approach in both challenging grid-world environments and preliminary Gazebo experiments. A video of our experiments can be found at: https://goo.gl/G2Vu5y.

A video of our experiments can be found at: https://goo.gl/G2Vu5y

References in corpus (8)

• FlowNet: Learning Optical Flow with Convolutional Networks
• Learning to Navigate in Complex Environments
• Target-driven Visual Navigation in Indoor Scenes using Deep Reinforcement Learning
• Neural Map: Structured Memory for Deep Reinforcement Learning
• Unifying Map and Landmark Based Representations for Visual Navigation
• Playing Doom with SLAM-Augmented Deep Reinforcement Learning
• Active Neural Localization
• Deep Reinforcement Learning with Successor Features for Navigation across Similar Environments

Cited by in corpus (42)

• Lifelong Federated Reinforcement Learning: A Learning Architecture for Navigation in Cloud Robotic Systems
• Curiosity-driven Exploration for Mapless Navigation with Deep Reinforcement Learning
• A Survey of Deep Network Solutions for Learning Control in Robotics: From Reinforcement to Imitation
• Unifying Map and Landmark Based Representations for Visual Navigation
• Learning to Navigate in Cities Without a Map
• Learning Exploration Policies for Navigation
• Memory Augmented Control Networks
• Learning Deployable Navigation Policies at Kilometer Scale from a Single Traversal
• Learning Robotic Navigation from Experience: Principles, Methods, and Recent Results
• Navigate-and-Seek: a Robotics Framework for People Localization in Agricultural Environments
• On Reward Shaping for Mobile Robot Navigation: A Reinforcement Learning and SLAM Based Approach
• Collaborative Visual Navigation
• Scene Memory Transformer for Embodied Agents in Long-Horizon Tasks
• VR-Goggles for Robots: Real-to-sim Domain Adaptation for Visual Control
• Value Propagation Networks
• Learning models for visual 3D localization with implicit mapping
• Shaping Belief States with Generative Environment Models for RL
• To Learn or Not to Learn: Analyzing the Role of Learning for Navigation in Virtual Environments
• Deep Context Maps: Agent Trajectory Prediction using Location-specific Latent Maps
• Deep Learning for Embodied Vision Navigation: A Survey
• Topological Planning with Transformers for Vision-and-Language Navigation
• Simultaneous Mapping and Target Driven Navigation
• Learning to Visually Navigate in Photorealistic Environments Without any Supervision
• MVP: Unified Motion and Visual Self-Supervised Learning for Large-Scale Robotic Navigation
• Graph Attention Memory for Visual Navigation
• Reinforcement Learning-Based Coverage Path Planning with Implicit Cellular Decomposition
• EMPNet: Neural Localisation and Mapping Using Embedded Memory Points
• SGoLAM: Simultaneous Goal Localization and Mapping for Multi-Object Goal Navigation
• SOON: Scenario Oriented Object Navigation with Graph-based Exploration
• Learning Object-conditioned Exploration using Distributed Soft Actor Critic
• Cross-View Policy Learning for Street Navigation
• SegVisRL: Visuomotor Development for a Lunar Rover for Hazard Avoidance using Camera Images
• Learning Efficient Multi-Agent Cooperative Visual Exploration
• Advances in Inference and Representation for Simultaneous Localization and Mapping
• Deep Visual Odometry with Adaptive Memory
• A short note on the decision tree based neural turing machine
• Localising In Complex Scenes Using Balanced Adversarial Adaptation
• Incremental Scene Synthesis
• Building Intelligent Autonomous Navigation Agents
• A Self-Supervised Auxiliary Loss for Deep RL in Partially Observable Settings
• Learning Robot Exploration Strategy with 4D Point-Clouds-like Information as Observations
• Variational State-Space Models for Localisation and Dense 3D Mapping in 6 DoF