Rescuing Deep Hashing from Dead Bits Problem

Abstract

Deep hashing methods have shown great retrieval accuracy and efficiency in large-scale image retrieval. How to optimize discrete hash bits is always the focus in deep hashing methods. A common strategy in these methods is to adopt an activation function, e.g. $\mathrm{sigmoid}(\cdot )$ or $\tanh (\cdot )$, and minimize a quantization loss to approximate discrete values. However, this paradigm may make more and more hash bits stuck into the wrong saturated area of the activation functions and never escaped. We call this problem "Dead Bits Problem~(DBP)". Besides, the existing quantization loss will aggravate DBP as well. In this paper, we propose a simple but effective gradient amplifier which acts before activation functions to alleviate DBP. Moreover, we devise an error-aware quantization loss to further alleviate DBP. It avoids the negative effect of quantization loss based on the similarity between two images. The proposed gradient amplifier and error-aware quantization loss are compatible with a variety of deep hashing methods. Experimental results on three datasets demonstrate the efficiency of the proposed gradient amplifier and the error-aware quantization loss.

References in corpus (7)

• PyTorch: An Imperative Style, High-Performance Deep Learning Library
• SGDR: Stochastic Gradient Descent with Warm Restarts
• Simultaneous Feature Learning and Hash Coding with Deep Neural Networks
• Supervised Discrete Hashing
• Deep Semantic Ranking Based Hashing for Multi-Label Image Retrieval
• Feature Learning based Deep Supervised Hashing with Pairwise Labels
• Deep Momentum Uncertainty Hashing