UR2KiD: Unifying Retrieval, Keypoint Detection, and Keypoint Description without Local Correspondence Supervision

Abstract

In this paper, we explore how three related tasks, namely keypoint detection, description, and image retrieval can be jointly tackled using a single unified framework, which is trained without the need of training data with point to point correspondences. By leveraging diverse information from sequential layers of a standard ResNet-based architecture, we are able to extract keypoints and descriptors that encode local information using generic techniques such as local activation norms, channel grouping and dropping, and self-distillation. Subsequently, global information for image retrieval is encoded in an end-to-end pipeline, based on pooling of the aforementioned local responses. In contrast to previous methods in local matching, our method does not depend on pointwise/pixelwise correspondences, and requires no such supervision at all i.e. no depth-maps from an SfM model nor manually created synthetic affine transformations. We illustrate that this simple and direct paradigm, is able to achieve very competitive results against the state-of-the-art methods in various challenging benchmark conditions such as viewpoint changes, scale changes, and day-night shifting localization.

References in corpus (11)

• Distilling the Knowledge in a Neural Network
• EfficientNet: Rethinking Model Scaling for Convolutional Neural Networks
• R2D2: Repeatable and Reliable Detector and Descriptor
• PN-Net: Conjoined Triple Deep Network for Learning Local Image Descriptors
• Key.Net: Keypoint Detection by Handcrafted and Learned CNN Filters
• Large-Scale Image Retrieval with Attentive Deep Local Features
• Unsupervised Semantic-based Aggregation of Deep Convolutional Features
• Benchmarking 6DOF Outdoor Visual Localization in Changing Conditions
• ContextDesc: Local Descriptor Augmentation with Cross-Modality Context
• Local Descriptors Optimized for Average Precision
• DSAC - Differentiable RANSAC for Camera Localization

Cited by in corpus (7)

• Reference Pose Generation for Long-term Visual Localization via Learned Features and View Synthesis
• AdaLAM: Revisiting Handcrafted Outlier Detection
• CNN-based local features for navigation near an asteroid
• SOLD2: Self-supervised Occlusion-aware Line Description and Detection
• Wide-Baseline Multi-Camera Calibration using Person Re-Identification
• Soft Expectation and Deep Maximization for Image Feature Detection
• Learning Local Features with Context Aggregation for Visual Localization