Faithful Multimodal Explanation for Visual Question Answering
arXiv:1809.02805
Abstract
AI systems' ability to explain their reasoning is critical to their utility and trustworthiness. Deep neural networks have enabled significant progress on many challenging problems such as visual question answering (VQA). However, most of them are opaque black boxes with limited explanatory capability. This paper presents a novel approach to developing a high-performing VQA system that can elucidate its answers with integrated textual and visual explanations that faithfully reflect important aspects of its underlying reasoning while capturing the style of comprehensible human explanations. Extensive experimental evaluation demonstrates the advantages of this approach compared to competing methods with both automatic evaluation metrics and human evaluation metrics.
arXiv commentIn ACL 2019 BlackboxNLP workshop
References in corpus (17)
- Least Angle Regression
- Discussion of "Least angle regression" by Efron et al
- Discussion of "Least angle regression" by Efron et al
- Discussion of "Least angle regression" by Efron et al
- Discussion of "Least angle regression" by Efron et al
- Discussion of "Least angle regression" by Efron et al
- Discussion of "Least angle regression" by Efron et al
- Discussion of "Least angle regression" by Efron et al
- Discussion of "Least angle regression" by Efron et al
- Rejoinder to "Least angle regression" by Efron et al
- Learning Phrase Representations using RNN Encoder-Decoder for Statistical Machine Translation
- Hierarchical Question-Image Co-Attention for Visual Question Answering
- VQA: Visual Question Answering
- Self-Critical Reasoning for Robust Visual Question Answering
- Constituency Parsing with a Self-Attentive Encoder
- Joint Image Captioning and Question Answering
- Grounding Visual Explanations
Cited by in corpus (5)
- Explainable Activity Recognition for Smart Home Systems
- Generating Question Relevant Captions to Aid Visual Question Answering
- Faithfully Explainable Recommendation via Neural Logic Reasoning
- Designing Multimodal Datasets for NLP Challenges
- Hidden State Guidance: Improving Image Captioning using An Image Conditioned Autoencoder