Self-Supervised Place Recognition by Refining Temporal and Featural Pseudo Labels from Panoramic Data

Abstract

Visual place recognition (VPR) using deep networks has achieved state-of-the-art performance. However, most of them require a training set with ground truth sensor poses to obtain positive and negative samples of each observation's spatial neighborhood for supervised learning. When such information is unavailable, temporal neighborhoods from a sequentially collected data stream could be exploited for self-supervised training, although we find its performance suboptimal. Inspired by noisy label learning, we propose a novel self-supervised framework named TF-VPR that uses temporal neighborhoods and learnable feature neighborhoods to discover unknown spatial neighborhoods. Our method follows an iterative training paradigm which alternates between: (1) representation learning with data augmentation, (2) positive set expansion to include the current feature space neighbors, and (3) positive set contraction via geometric verification. We conduct auto-labeling and generalization tests on both simulated and real datasets, with either RGB images or point clouds as inputs. The results show that our method outperforms self-supervised baselines in recall rate, robustness, and heading diversity, a novel metric we propose for VPR. Our code and datasets can be found at https://ai4ce.github.io/TF-VPR/

References in corpus (13)

• Adam: A Method for Stochastic Optimization
• Representation Learning with Contrastive Predictive Coding
• Training Deep Neural Networks on Noisy Labels with Bootstrapping
• AugMix: A Simple Data Processing Method to Improve Robustness and Uncertainty
• Prototypical Contrastive Learning of Unsupervised Representations
• OpenVSLAM: A Versatile Visual SLAM Framework
• VPR-Bench: An Open-Source Visual Place Recognition Evaluation Framework with Quantifiable Viewpoint and Appearance Change
• Gibson Env: Real-World Perception for Embodied Agents
• PADLoC: LiDAR-Based Deep Loop Closure Detection and Registration Using Panoptic Attention
• KITTI-360: A Novel Dataset and Benchmarks for Urban Scene Understanding in 2D and 3D
• Self-Supervised Domain Calibration and Uncertainty Estimation for Place Recognition
• Local positional graphs and attentive local features for a data and runtime-efficient hierarchical place recognition pipeline
• NYU-VPR: Long-Term Visual Place Recognition Benchmark with View Direction and Data Anonymization Influences