Bags of Local Convolutional Features for Scalable Instance Search

Abstract

This work proposes a simple instance retrieval pipeline based on encoding the convolutional features of CNN using the bag of words aggregation scheme (BoW). Assigning each local array of activations in a convolutional layer to a visual word produces an \textit{assignment map}, a compact representation that relates regions of an image with a visual word. We use the assignment map for fast spatial reranking, obtaining object localizations that are used for query expansion. We demonstrate the suitability of the BoW representation based on local CNN features for instance retrieval, achieving competitive performance on the Oxford and Paris buildings benchmarks. We show that our proposed system for CNN feature aggregation with BoW outperforms state-of-the-art techniques using sum pooling at a subset of the challenging TRECVid INS benchmark.

Preprint of a short paper accepted in the ACM International Conference on Multimedia Retrieval (ICMR) 2016 (New York City, NY, USA)

References in corpus (5)

• Very Deep Convolutional Networks for Large-Scale Image Recognition
• Caffe: Convolutional Architecture for Fast Feature Embedding
• Particular object retrieval with integral max-pooling of CNN activations
• Exploiting Local Features from Deep Networks for Image Retrieval
• Cross-dimensional Weighting for Aggregated Deep Convolutional Features

Cited by in corpus (10)

• A Decade Survey of Content Based Image Retrieval using Deep Learning
• Learning Non-Metric Visual Similarity for Image Retrieval
• SIFT Meets CNN: A Decade Survey of Instance Retrieval
• Evaluating Contrastive Models for Instance-based Image Retrieval
• An accurate retrieval through R-MAC+ descriptors for landmark recognition
• The 2021 Image Similarity Dataset and Challenge
• Semantic Reinforced Attention Learning for Visual Place Recognition
• Change Detection under Global Viewpoint Uncertainty
• Combating Ambiguity for Hash-code Learning in Medical Instance Retrieval
• Analyzing the Dependency of ConvNets on Spatial Information