EMPNet: Neural Localisation and Mapping Using Embedded Memory Points
arXiv:1907.13268
Abstract
Continuously estimating an agent's state space and a representation of its surroundings has proven vital towards full autonomy. A shared common ground among systems which successfully achieve this feat is the integration of previously encountered observations into the current state being estimated. This necessitates the use of a memory module for incorporating previously visited states whilst simultaneously offering an internal representation of the observed environment. In this work we develop a memory module which contains rigidly aligned point-embeddings that represent a coherent scene structure acquired from an RGB-D sequence of observations. The point-embeddings are extracted using modern convolutional neural network architectures, and alignment is performed by computing a dense correspondence matrix between a new observation and the current embeddings residing in the memory module. The whole framework is end-to-end trainable, resulting in a recurrent joint optimisation of the point-embeddings contained in the memory. This process amplifies the shared information across states, providing increased robustness and accuracy. We show significant improvement of our method across a set of experiments performed on the synthetic VIZDoom environment and a real world Active Vision Dataset.
Accepted at ICCV 2019
References in corpus (10)
- Adam: A Method for Stochastic Optimization
- Very Deep Convolutional Networks for Large-Scale Image Recognition
- Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks
- Depth Map Prediction from a Single Image using a Multi-Scale Deep Network
- DeepVO: Towards End-to-End Visual Odometry with Deep Recurrent Convolutional Neural Networks
- Deep Predictive Coding Networks for Video Prediction and Unsupervised Learning
- Neural SLAM: Learning to Explore with External Memory
- Neural Map: Structured Memory for Deep Reinforcement Learning
- Just-in-Time Reconstruction: Inpainting Sparse Maps using Single View Depth Predictors as Priors
- ENG: End-to-end Neural Geometry for Robust Depth and Pose Estimation using CNNs