Image-based localization using LSTMs for structured feature correlation

Abstract

In this work we propose a new CNN+LSTM architecture for camera pose regression for indoor and outdoor scenes. CNNs allow us to learn suitable feature representations for localization that are robust against motion blur and illumination changes. We make use of LSTM units on the CNN output, which play the role of a structured dimensionality reduction on the feature vector, leading to drastic improvements in localization performance. We provide extensive quantitative comparison of CNN-based and SIFT-based localization methods, showing the weaknesses and strengths of each. Furthermore, we present a new large-scale indoor dataset with accurate ground truth from a laser scanner. Experimental results on both indoor and outdoor public datasets show our method outperforms existing deep architectures, and can localize images in hard conditions, e.g., in the presence of mostly textureless surfaces, where classic SIFT-based methods fail.

References in corpus (9)

• Very Deep Convolutional Networks for Large-Scale Image Recognition
• Sequence to Sequence Learning with Neural Networks
• Deep Residual Learning for Image Recognition
• Going Deeper with Convolutions
• PlaNet - Photo Geolocation with Convolutional Neural Networks
• Convolutional Oriented Boundaries
• Pushing the Boundaries of Boundary Detection using Deep Learning
• Semantic Object Parsing with Local-Global Long Short-Term Memory
• DSAC - Differentiable RANSAC for Camera Localization

Cited by in corpus (6)

• Deep EndoVO: A Recurrent Convolutional Neural Network (RCNN) based Visual Odometry Approach for Endoscopic Capsule Robots
• Backtracking Regression Forests for Accurate Camera Relocalization
• DeLS-3D: Deep Localization and Segmentation with a 3D Semantic Map
• Cascaded Parallel Filtering for Memory-Efficient Image-Based Localization
• A Generative Map for Image-based Camera Localization
• Euler angles based loss function for camera relocalization with Deep learning