Towards Transparent AI Systems: Interpreting Visual Question Answering Models

Abstract

Deep neural networks have shown striking progress and obtained state-of-the-art results in many AI research fields in the recent years. However, it is often unsatisfying to not know why they predict what they do. In this paper, we address the problem of interpreting Visual Question Answering (VQA) models. Specifically, we are interested in finding what part of the input (pixels in images or words in questions) the VQA model focuses on while answering the question. To tackle this problem, we use two visualization techniques -- guided backpropagation and occlusion -- to find important words in the question and important regions in the image. We then present qualitative and quantitative analyses of these importance maps. We found that even without explicit attention mechanisms, VQA models may sometimes be implicitly attending to relevant regions in the image, and often to appropriate words in the question.

References in corpus (2)

• Sequence to Sequence Learning with Neural Networks
• Large-scale Simple Question Answering with Memory Networks

Cited by in corpus (11)

• Levels of explainable artificial intelligence for human-aligned conversational explanations
• Explainable Artificial Intelligence: a Systematic Review
• Attentive Explanations: Justifying Decisions and Pointing to the Evidence
• Interpretable Convolutional Neural Networks
• Finding the Evidence: Localization-aware Answer Prediction for Text Visual Question Answering
• Examining CNN Representations with respect to Dataset Bias
• An Entropy Clustering Approach for Assessing Visual Question Difficulty
• Interpretable CNNs for Object Classification
• Generating Rationales in Visual Question Answering
• A First Look: Towards Explainable TextVQA Models via Visual and Textual Explanations
• Information-theoretic Evolution of Model Agnostic Global Explanations