Deep Image Retrieval: Learning global representations for image search

Abstract

We propose a novel approach for instance-level image retrieval. It produces a global and compact fixed-length representation for each image by aggregating many region-wise descriptors. In contrast to previous works employing pre-trained deep networks as a black box to produce features, our method leverages a deep architecture trained for the specific task of image retrieval. Our contribution is twofold: (i) we leverage a ranking framework to learn convolution and projection weights that are used to build the region features; and (ii) we employ a region proposal network to learn which regions should be pooled to form the final global descriptor. We show that using clean training data is key to the success of our approach. To that aim, we use a large scale but noisy landmark dataset and develop an automatic cleaning approach. The proposed architecture produces a global image representation in a single forward pass. Our approach significantly outperforms previous approaches based on global descriptors on standard datasets. It even surpasses most prior works based on costly local descriptor indexing and spatial verification. Additional material is available at www.xrce.xerox.com/Deep-Image-Retrieval.

ECCV 2016 version + additional results

References in corpus (3)

• Very Deep Convolutional Networks for Large-Scale Image Recognition
• Deep Fragment Embeddings for Bidirectional Image Sentence Mapping
• Multiple Measurements and Joint Dimensionality Reduction for Large Scale Image Search with Short Vectors - Extended Version

Cited by in corpus (26)

• Recent Advance in Content-based Image Retrieval: A Literature Survey
• Re-ID done right: towards good practices for person re-identification
• Generalisation and Sharing in Triplet Convnets for Sketch based Visual Search
• Large-Scale Image Retrieval with Attentive Deep Local Features
• Unsupervised Semantic-based Aggregation of Deep Convolutional Features
• Are we done with object recognition? The iCub robot's perspective
• A deep architecture for unified aesthetic prediction
• Exploiting Deep Features for Remote Sensing Image Retrieval: A Systematic Investigation
• Co-salient Object Detection Based on Deep Saliency Networks and Seed Propagation over an Integrated Graph
• Revisiting Image-Language Networks for Open-ended Phrase Detection
• Intermediate Deep Feature Compression: the Next Battlefield of Intelligent Sensing
• DenseAttentionSeg: Segment Hands from Interacted Objects Using Depth Input
• Link and code: Fast indexing with graphs and compact regression codes
• Selective Deep Convolutional Features for Image Retrieval
• End-to-End Supervised Product Quantization for Image Search and Retrieval
• Generalization in Metric Learning: Should the Embedding Layer be the Embedding Layer?
• An accurate retrieval through R-MAC+ descriptors for landmark recognition
• Iterative Manifold Embedding Layer Learned by Incomplete Data for Large-scale Image Retrieval
• Loop Closure Detection with RGB-D Feature Pyramid Siamese Networks
• Adversarial Soft-detection-based Aggregation Network for Image Retrieval
• Accurate and Efficient Similarity Search for Large Scale Face Recognition
• An evaluation of large-scale methods for image instance and class discovery
• One-Shot Fine-Grained Instance Retrieval
• Composing Text and Image for Image Retrieval - An Empirical Odyssey
• Learning a Complete Image Indexing Pipeline
• Push it to the Limit: Discover Edge-Cases in Image Data with Autoencoders