Exploiting Local Features from Deep Networks for Image Retrieval

Abstract

Deep convolutional neural networks have been successfully applied to image classification tasks. When these same networks have been applied to image retrieval, the assumption has been made that the last layers would give the best performance, as they do in classification. We show that for instance-level image retrieval, lower layers often perform better than the last layers in convolutional neural networks. We present an approach for extracting convolutional features from different layers of the networks, and adopt VLAD encoding to encode features into a single vector for each image. We investigate the effect of different layers and scales of input images on the performance of convolutional features using the recent deep networks OxfordNet and GoogLeNet. Experiments demonstrate that intermediate layers or higher layers with finer scales produce better results for image retrieval, compared to the last layer. When using compressed 128-D VLAD descriptors, our method obtains state-of-the-art results and outperforms other VLAD and CNN based approaches on two out of three test datasets. Our work provides guidance for transferring deep networks trained on image classification to image retrieval tasks.

CVPR DeepVision Workshop 2015

References in corpus (6)

• Very Deep Convolutional Networks for Large-Scale Image Recognition
• Caffe: Convolutional Architecture for Fast Feature Embedding
• Going Deeper with Convolutions
• OverFeat: Integrated Recognition, Localization and Detection using Convolutional Networks
• CNN Features off-the-shelf: an Astounding Baseline for Recognition
• A Discriminative CNN Video Representation for Event Detection

Cited by in corpus (22)

• Bags of Local Convolutional Features for Scalable Instance Search
• Good Practice in CNN Feature Transfer
• Large-Scale Image Retrieval with Attentive Deep Local Features
• NoScope: Optimizing Neural Network Queries over Video at Scale
• Exploiting Deep Features for Remote Sensing Image Retrieval: A Systematic Investigation
• Deep Region Hashing for Efficient Large-scale Instance Search from Images
• SIFT Meets CNN: A Decade Survey of Instance Retrieval
• Crowd Counting via Weighted VLAD on Dense Attribute Feature Maps
• Centroid Based Concept Learning for RGB-D Indoor Scene Classification
• Local Feature Detectors, Descriptors, and Image Representations: A Survey
• Unsupervised Category Discovery via Looped Deep Pseudo-Task Optimization Using a Large Scale Radiology Image Database
• Unsupervised Joint Mining of Deep Features and Image Labels for Large-scale Radiology Image Categorization and Scene Recognition
• Instance Search via Instance Level Segmentation and Feature Representation
• Exploiting Local Feature Patterns for Unsupervised Domain Adaptation
• Learning a Robust Representation via a Deep Network on Symmetric Positive Definite Manifolds
• An accurate retrieval through R-MAC+ descriptors for landmark recognition
• Effective Image Retrieval via Multilinear Multi-index Fusion
• Generic Instance Search and Re-identification from One Example via Attributes and Categories
• One-Shot Fine-Grained Instance Retrieval
• Automatic Synchronization of Multi-User Photo Galleries
• Towards Visual Feature Translation
• Multiple VLAD encoding of CNNs for image classification