Self-supervising Fine-grained Region Similarities for Large-scale Image Localization

Abstract

The task of large-scale retrieval-based image localization is to estimate the geographical location of a query image by recognizing its nearest reference images from a city-scale dataset. However, the general public benchmarks only provide noisy GPS labels associated with the training images, which act as weak supervisions for learning image-to-image similarities. Such label noise prevents deep neural networks from learning discriminative features for accurate localization. To tackle this challenge, we propose to self-supervise image-to-region similarities in order to fully explore the potential of difficult positive images alongside their sub-regions. The estimated image-to-region similarities can serve as extra training supervision for improving the network in generations, which could in turn gradually refine the fine-grained similarities to achieve optimal performance. Our proposed self-enhanced image-to-region similarity labels effectively deal with the training bottleneck in the state-of-the-art pipelines without any additional parameters or manual annotations in both training and inference. Our method outperforms state-of-the-arts on the standard localization benchmarks by noticeable margins and shows excellent generalization capability on multiple image retrieval datasets.

arXiv commentAccepted in ECCV 2020 (Spotlight), code is available at https://github.com/yxgeee/SFRS

References in corpus (9)

• Very Deep Convolutional Networks for Large-Scale Image Recognition
• Unsupervised Representation Learning by Predicting Image Rotations
• Deep Clustering for Unsupervised Learning of Visual Features
• Born Again Neural Networks
• PlaNet - Photo Geolocation with Convolutional Neural Networks
• Mutual Mean-Teaching: Pseudo Label Refinery for Unsupervised Domain Adaptation on Person Re-identification
• FD-GAN: Pose-guided Feature Distilling GAN for Robust Person Re-identification
• Rotate-and-Render: Unsupervised Photorealistic Face Rotation from Single-View Images
• Discriminability Distillation in Group Representation Learning

Cited by in corpus (3)

• Deep Homography Estimation for Visual Place Recognition
• Robot Perception enables Complex Navigation Behavior via Self-Supervised Learning
• VS-Net: Voting with Segmentation for Visual Localization