A Decade Survey of Content Based Image Retrieval using Deep Learning

Abstract

The content based image retrieval aims to find the similar images from a large scale dataset against a query image. Generally, the similarity between the representative features of the query image and dataset images is used to rank the images for retrieval. In early days, various hand designed feature descriptors have been investigated based on the visual cues such as color, texture, shape, etc. that represent the images. However, the deep learning has emerged as a dominating alternative of hand-designed feature engineering from a decade. It learns the features automatically from the data. This paper presents a comprehensive survey of deep learning based developments in the past decade for content based image retrieval. The categorization of existing state-of-the-art methods from different perspectives is also performed for greater understanding of the progress. The taxonomy used in this survey covers different supervision, different networks, different descriptor type and different retrieval type. A performance analysis is also performed using the state-of-the-art methods. The insights are also presented for the benefit of the researchers to observe the progress and to make the best choices. The survey presented in this paper will help in further research progress in image retrieval using deep learning.

Published by IEEE Transactions on Circuits and Systems for Video Technology

References in corpus (30)

• Very Deep Convolutional Networks for Large-Scale Image Recognition
• Simultaneous Feature Learning and Hash Coding with Deep Neural Networks
• Particular object retrieval with integral max-pooling of CNN activations
• A Survey on Metric Learning for Feature Vectors and Structured Data
• Bit-Scalable Deep Hashing with Regularized Similarity Learning for Image Retrieval and Person Re-identification
• Medical Image Retrieval using Deep Convolutional Neural Network
• Gated Feedback Recurrent Neural Networks
• Supervised Learning of Semantics-Preserving Hash via Deep Convolutional Neural Networks
• Hierarchical Multiscale Recurrent Neural Networks
• Doodle to Search: Practical Zero-Shot Sketch-based Image Retrieval
• Learning Local Image Descriptors with Deep Siamese and Triplet Convolutional Networks by Minimising Global Loss Functions
• SSDH: Semi-supervised Deep Hashing for Large Scale Image Retrieval
• Deep Supervised Discrete Hashing
• Recent Advance in Content-based Image Retrieval: A Literature Survey
• Bags of Local Convolutional Features for Scalable Instance Search
• Instance-Aware Hashing for Multi-Label Image Retrieval
• Binary Generative Adversarial Networks for Image Retrieval
• Semi-Heterogeneous Three-Way Joint Embedding Network for Sketch-Based Image Retrieval
• Average Biased ReLU Based CNN Descriptor for Improved Face Retrieval
• BinGAN: Learning Compact Binary Descriptors with a Regularized GAN
• Siamese Network of Deep Fisher-Vector Descriptors for Image Retrieval
• A Survey on Deep Hashing Methods
• Unsupervised Part-based Weighting Aggregation of Deep Convolutional Features for Image Retrieval
• Co-occurrence of deep convolutional features for image search
• Simultaneous Feature Aggregating and Hashing for Compact Binary Code Learning
• Group Invariant Deep Representations for Image Instance Retrieval
• Embarrassingly Simple Binary Representation Learning
• Asymmetric Deep Semantic Quantization for Image Retrieval
• Convolutional Patch Representations for Image Retrieval: an Unsupervised Approach
• SSAH: Semi-supervised Adversarial Deep Hashing with Self-paced Hard Sample Generation

Cited by in corpus (9)

• Guided Unsupervised Learning by Subaperture Decomposition for Ocean SAR Image Retrieval
• One Loss for All: Deep Hashing with a Single Cosine Similarity based Learning Objective
• Learning Image Representations for Content Based Image Retrieval of Radiotherapy Treatment Plans
• Local Concept Embeddings for Analysis of Concept Distributions in Vision DNN Feature Spaces
• Scalable Visual Attribute Extraction through Hidden Layers of a Residual ConvNet
• A multimodal deep learning framework for scalable content based visual media retrieval
• Frequency Disentangled Residual Network
• PrivateMail: Supervised Manifold Learning of Deep Features With Differential Privacy for Image Retrieval
• RFHNet: Relational and Frequency-Aware Hashing Network for Large-Scale Fine-Grained Food Image Retrieval