SIFT Meets CNN: A Decade Survey of Instance Retrieval

Abstract

In the early days, content-based image retrieval (CBIR) was studied with global features. Since 2003, image retrieval based on local descriptors (de facto SIFT) has been extensively studied for over a decade due to the advantage of SIFT in dealing with image transformations. Recently, image representations based on the convolutional neural network (CNN) have attracted increasing interest in the community and demonstrated impressive performance. Given this time of rapid evolution, this article provides a comprehensive survey of instance retrieval over the last decade. Two broad categories, SIFT-based and CNN-based methods, are presented. For the former, according to the codebook size, we organize the literature into using large/medium-sized/small codebooks. For the latter, we discuss three lines of methods, i.e., using pre-trained or fine-tuned CNN models, and hybrid methods. The first two perform a single-pass of an image to the network, while the last category employs a patch-based feature extraction scheme. This survey presents milestones in modern instance retrieval, reviews a broad selection of previous works in different categories, and provides insights on the connection between SIFT and CNN-based methods. After analyzing and comparing retrieval performance of different categories on several datasets, we discuss promising directions towards generic and specialized instance retrieval.

Accepted to IEEE Transactions on Pattern Analysis and Machine Intelligence

References in corpus (8)

• Descriptor Matching with Convolutional Neural Networks: a Comparison to SIFT
• Learning to Compare Image Patches via Convolutional Neural Networks
• Bags of Local Convolutional Features for Scalable Instance Search
• Good Practice in CNN Feature Transfer
• Exploiting Local Features from Deep Networks for Image Retrieval
• Generalized Max Pooling
• Object Level Deep Feature Pooling for Compact Image Representation
• End-to-end Learning of Deep Visual Representations for Image Retrieval

Cited by in corpus (8)

• Person Re-identification: Past, Present and Future
• A Discriminatively Learned CNN Embedding for Person Re-identification
• Unlabeled Samples Generated by GAN Improve the Person Re-identification Baseline in vitro
• Re-ranking Person Re-identification with k-reciprocal Encoding
• Large-Scale Image Retrieval with Attentive Deep Local Features
• A New Evaluation Protocol and Benchmarking Results for Extendable Cross-media Retrieval
• Change Detection under Global Viewpoint Uncertainty
• A framework with updateable joint images re-ranking for Person Re-identification