Towards Interpretable and Robust Hand Detection via Pixel-wise Prediction

Abstract

The lack of interpretability of existing CNN-based hand detection methods makes it difficult to understand the rationale behind their predictions. In this paper, we propose a novel neural network model, which introduces interpretability into hand detection for the first time. The main improvements include: (1) Detect hands at pixel level to explain what pixels are the basis for its decision and improve transparency of the model. (2) The explainable Highlight Feature Fusion block highlights distinctive features among multiple layers and learns discriminative ones to gain robust performance. (3) We introduce a transparent representation, the rotation map, to learn rotation features instead of complex and non-transparent rotation and derotation layers. (4) Auxiliary supervision accelerates the training process, which saves more than 10 hours in our experiments. Experimental results on the VIVA and Oxford hand detection and tracking datasets show competitive accuracy of our method compared with state-of-the-art methods with higher speed.

Accepted to Pattern Recognition

References in corpus (7)

• Very Deep Convolutional Networks for Large-Scale Image Recognition
• Simple Online and Realtime Tracking
• Fully Convolutional Networks for Semantic Segmentation
• UnitBox: An Advanced Object Detection Network
• Explaining NonLinear Classification Decisions with Deep Taylor Decomposition
• DenseBox: Unifying Landmark Localization with End to End Object Detection
• Brain Tumor Segmentation Based on Refined Fully Convolutional Neural Networks with A Hierarchical Dice Loss

Cited by in corpus (2)

• Counterfactual Explanation Based on Gradual Construction for Deep Networks
• FaceTouch: Detecting hand-to-face touch with supervised contrastive learning to assist in tracing infectious disease