Improving Interpretability of Deep Neural Networks with Semantic Information

Abstract

Interpretability of deep neural networks (DNNs) is essential since it enables users to understand the overall strengths and weaknesses of the models, conveys an understanding of how the models will behave in the future, and how to diagnose and correct potential problems. However, it is challenging to reason about what a DNN actually does due to its opaque or black-box nature. To address this issue, we propose a novel technique to improve the interpretability of DNNs by leveraging the rich semantic information embedded in human descriptions. By concentrating on the video captioning task, we first extract a set of semantically meaningful topics from the human descriptions that cover a wide range of visual concepts, and integrate them into the model with an interpretive loss. We then propose a prediction difference maximization algorithm to interpret the learned features of each neuron. Experimental results demonstrate its effectiveness in video captioning using the interpretable features, which can also be transferred to video action recognition. By clearly understanding the learned features, users can easily revise false predictions via a human-in-the-loop procedure.

arXiv commentTo appear in CVPR 2017

References in corpus (17)

• Empirical Evaluation of Gated Recurrent Neural Networks on Sequence Modeling
• Show, Attend and Tell: Neural Image Caption Generation with Visual Attention
• ADADELTA: An Adaptive Learning Rate Method
• End to End Learning for Self-Driving Cars
• Going Deeper with Convolutions
• Visualizing and Understanding Recurrent Networks
• Rich feature hierarchies for accurate object detection and semantic segmentation
• Visualizing Deep Neural Network Decisions: Prediction Difference Analysis
• An Empirical Evaluation of Deep Learning on Highway Driving
• Learning Deep Features for Discriminative Localization
• Sequence to Sequence -- Video to Text
• Describing Videos by Exploiting Temporal Structure
• Multifaceted Feature Visualization: Uncovering the Different Types of Features Learned By Each Neuron in Deep Neural Networks
• Delving Deeper into Convolutional Networks for Learning Video Representations
• Video Paragraph Captioning Using Hierarchical Recurrent Neural Networks
• Jointly Modeling Embedding and Translation to Bridge Video and Language
• Towards Better Analysis of Deep Convolutional Neural Networks

Cited by in corpus (6)

• Deep Reinforcement Learning: An Overview
• Towards Interpretable Deep Neural Networks by Leveraging Adversarial Examples
• Regional Multi-scale Approach for Visually Pleasing Explanations of Deep Neural Networks
• Video Captioning with Guidance of Multimodal Latent Topics
• Learning Less-Overlapping Representations
• Understanding Misclassifications by Attributes