Deep Sketch Hashing: Fast Free-hand Sketch-Based Image Retrieval

Abstract

Free-hand sketch-based image retrieval (SBIR) is a specific cross-view retrieval task, in which queries are abstract and ambiguous sketches while the retrieval database is formed with natural images. Work in this area mainly focuses on extracting representative and shared features for sketches and natural images. However, these can neither cope well with the geometric distortion between sketches and images nor be feasible for large-scale SBIR due to the heavy continuous-valued distance computation. In this paper, we speed up SBIR by introducing a novel binary coding method, named \textbf{Deep Sketch Hashing} (DSH), where a semi-heterogeneous deep architecture is proposed and incorporated into an end-to-end binary coding framework. Specifically, three convolutional neural networks are utilized to encode free-hand sketches, natural images and, especially, the auxiliary sketch-tokens which are adopted as bridges to mitigate the sketch-image geometric distortion. The learned DSH codes can effectively capture the cross-view similarities as well as the intrinsic semantic correlations between different categories. To the best of our knowledge, DSH is the first hashing work specifically designed for category-level SBIR with an end-to-end deep architecture. The proposed DSH is comprehensively evaluated on two large-scale datasets of TU-Berlin Extension and Sketchy, and the experiments consistently show DSH's superior SBIR accuracies over several state-of-the-art methods, while achieving significantly reduced retrieval time and memory footprint.

arXiv commentThis paper will appear as a spotlight paper in CVPR2017

References in corpus (6)

• Caffe: Convolutional Architecture for Fast Feature Embedding
• Supervised Discrete Hashing
• Sketch-based 3D Shape Retrieval using Convolutional Neural Networks
• Sketch-a-Net that Beats Humans
• Deep Cross-Modal Hashing
• Correlation Hashing Network for Efficient Cross-Modal Retrieval

Cited by in corpus (11)

• Learning Robust Representations via Multi-View Information Bottleneck
• Cross-Modal Hierarchical Modelling for Fine-Grained Sketch Based Image Retrieval
• Creative Sketch Generation
• SketchMate: Deep Hashing for Million-Scale Human Sketch Retrieval
• Zero-Shot Sketch-Image Hashing
• Deep Binaries: Encoding Semantic-Rich Cues for Efficient Textual-Visual Cross Retrieval
• Unsupervised Sketch-to-Photo Synthesis
• Instance-level Sketch-based Retrieval by Deep Triplet Classification Siamese Network
• Sketch-a-Classifier: Sketch-based Photo Classifier Generation
• Deep Shape Matching
• Deep Factorised Inverse-Sketching