Unifying Deep Local and Global Features for Image Search

Abstract

Image retrieval is the problem of searching an image database for items that are similar to a query image. To address this task, two main types of image representations have been studied: global and local image features. In this work, our key contribution is to unify global and local features into a single deep model, enabling accurate retrieval with efficient feature extraction. We refer to the new model as DELG, standing for DEep Local and Global features. We leverage lessons from recent feature learning work and propose a model that combines generalized mean pooling for global features and attentive selection for local features. The entire network can be learned end-to-end by carefully balancing the gradient flow between two heads -- requiring only image-level labels. We also introduce an autoencoder-based dimensionality reduction technique for local features, which is integrated into the model, improving training efficiency and matching performance. Comprehensive experiments show that our model achieves state-of-the-art image retrieval on the Revisited Oxford and Paris datasets, and state-of-the-art single-model instance-level recognition on the Google Landmarks dataset v2. Code and models are available at https://github.com/tensorflow/models/tree/master/research/delf .

ECCV'20 paper

References in corpus (4)

• R2D2: Repeatable and Reliable Detector and Descriptor
• LF-Net: Learning Local Features from Images
• Key.Net: Keypoint Detection by Handcrafted and Learned CNN Filters
• Large-scale Landmark Retrieval/Recognition under a Noisy and Diverse Dataset

Cited by in corpus (6)

• VPR-Bench: An Open-Source Visual Place Recognition Evaluation Framework with Quantifiable Viewpoint and Appearance Change
• Supporting large-scale image recognition with out-of-domain samples
• Efficient large-scale image retrieval with deep feature orthogonality and Hybrid-Swin-Transformers
• Retrieval and Localization with Observation Constraints
• Unifying Remote Sensing Image Retrieval and Classification with Robust Fine-tuning
• Connecting Images through Time and Sources: Introducing Low-data, Heterogeneous Instance Retrieval