On the Performance of ConvNet Features for Place Recognition

Abstract

After the incredible success of deep learning in the computer vision domain, there has been much interest in applying Convolutional Network (ConvNet) features in robotic fields such as visual navigation and SLAM. Unfortunately, there are fundamental differences and challenges involved. Computer vision datasets are very different in character to robotic camera data, real-time performance is essential, and performance priorities can be different. This paper comprehensively evaluates and compares the utility of three state-of-the-art ConvNets on the problems of particular relevance to navigation for robots; viewpoint-invariance and condition-invariance, and for the first time enables real-time place recognition performance using ConvNets with large maps by integrating a variety of existing (locality-sensitive hashing) and novel (semantic search space partitioning) optimization techniques. We present extensive experiments on four real world datasets cultivated to evaluate each of the specific challenges in place recognition. The results demonstrate that speed-ups of two orders of magnitude can be achieved with minimal accuracy degradation, enabling real-time performance. We confirm that networks trained for semantic place categorization also perform better at (specific) place recognition when faced with severe appearance changes and provide a reference for which networks and layers are optimal for different aspects of the place recognition problem.

References in corpus (2)

• Improving neural networks by preventing co-adaptation of feature detectors
• Convolutional Neural Network-based Place Recognition

Cited by in corpus (45)

• Deep EndoVO: A Recurrent Convolutional Neural Network (RCNN) based Visual Odometry Approach for Endoscopic Capsule Robots
• A comparison of Vector Symbolic Architectures
• A Survey on Deep Learning for Localization and Mapping: Towards the Age of Spatial Machine Intelligence
• Multi-Process Fusion: Visual Place Recognition Using Multiple Image Processing Methods
• Artificial Intelligence and its Role in Near Future
• A Survey of Deep Network Solutions for Learning Control in Robotics: From Reinforcement to Imitation
• Single-View Place Recognition under Seasonal Changes
• Training a Convolutional Neural Network for Appearance-Invariant Place Recognition
• Condition-Invariant Multi-View Place Recognition
• Unsupervised Learning Methods for Visual Place Recognition in Discretely and Continuously Changing Environments
• Levelling the Playing Field: A Comprehensive Comparison of Visual Place Recognition Approaches under Changing Conditions
• Learning Deep NBNN Representations for Robust Place Categorization
• Deep Learning a Grasp Function for Grasping under Gripper Pose Uncertainty
• Probabilistic Visual Place Recognition for Hierarchical Localization
• Benchmarking 6DOF Outdoor Visual Localization in Changing Conditions
• Panoramic Annular Localizer: Tackling the Variation Challenges of Outdoor Localization Using Panoramic Annular Images and Active Deep Descriptors
• Robotic Grasp Detection using Deep Convolutional Neural Networks
• Lightweight Unsupervised Deep Loop Closure
• A Survey of Deep Learning Techniques for Mobile Robot Applications
• Training recurrent networks to generate hypotheses about how the brain solves hard navigation problems
• Scan Context++: Structural Place Recognition Robust to Rotation and Lateral Variations in Urban Environments
• Self-localization from Images with Small Overlap
• SymbioLCD: Ensemble-Based Loop Closure Detection using CNN-Extracted Objects and Visual Bag-of-Words
• Omnidirectional CNN for Visual Place Recognition and Navigation
• DeepSeqSLAM: A Trainable CNN+RNN for Joint Global Description and Sequence-based Place Recognition
• Accurate Vision-based Vehicle Localization using Satellite Imagery
• MVP: Unified Motion and Visual Self-Supervised Learning for Large-Scale Robotic Navigation
• Semantically-Aware Attentive Neural Embeddings for Image-based Visual Localization
• A Multi-Domain Feature Learning Method for Visual Place Recognition
• Feature Map Filtering: Improving Visual Place Recognition with Convolutional Calibration
• CAMAL: Context-Aware Multi-layer Attention framework for Lightweight Environment Invariant Visual Place Recognition
• Sequence-Based Filtering for Visual Route-Based Navigation: Analysing the Benefits, Trade-offs and Design Choices
• Fast and Incremental Loop Closure Detection Using Proximity Graphs
• Sparse Optimization for Robust and Efficient Loop Closing
• CNN Feature boosted SeqSLAM for Real-Time Loop Closure Detection
• Satellite Image-based Localization via Learned Embeddings
• Developing efficient transfer learning strategies for robust scene recognition in mobile robotics using pre-trained convolutional neural networks
• Beyond ANN: Exploiting Structural Knowledge for Efficient Place Recognition
• LiPo-LCD: Combining Lines and Points for Appearance-based Loop Closure Detection
• Fault-Diagnosing SLAM for Varying Scale Change Detection
• Excavate Condition-invariant Space by Intrinsic Encoder
• Dark Reciprocal-Rank: Boosting Graph-Convolutional Self-Localization Network via Teacher-to-student Knowledge Transfer
• Learning Navigation by Visual Localization and Trajectory Prediction
• Place recognition survey: An update on deep learning approaches
• Improving Place Recognition Using Dynamic Object Detection