A Study on Multimodal and Interactive Explanations for Visual Question Answering
arXiv:2003.00431
Abstract
Explainability and interpretability of AI models is an essential factor affecting the safety of AI. While various explainable AI (XAI) approaches aim at mitigating the lack of transparency in deep networks, the evidence of the effectiveness of these approaches in improving usability, trust, and understanding of AI systems are still missing. We evaluate multimodal explanations in the setting of a Visual Question Answering (VQA) task, by asking users to predict the response accuracy of a VQA agent with and without explanations. We use between-subjects and within-subjects experiments to probe explanation effectiveness in terms of improving user prediction accuracy, confidence, and reliance, among other factors. The results indicate that the explanations help improve human prediction accuracy, especially in trials when the VQA system's answer is inaccurate. Furthermore, we introduce active attention, a novel method for evaluating causal attentional effects through intervention by editing attention maps. User explanation ratings are strongly correlated with human prediction accuracy and suggest the efficacy of these explanations in human-machine AI collaboration tasks.
http://ceur-ws.org/Vol-2560/paper44.pdf
References in corpus (9)
- VQA: Visual Question Answering
- Object Detectors Emerge in Deep Scene CNNs
- Multimodal Compact Bilinear Pooling for Visual Question Answering and Visual Grounding
- Pythia v0.1: the Winning Entry to the VQA Challenge 2018
- Show, Ask, Attend, and Answer: A Strong Baseline For Visual Question Answering
- How do Humans Understand Explanations from Machine Learning Systems? An Evaluation of the Human-Interpretability of Explanation
- It Takes Two to Tango: Towards Theory of AI's Mind
- Generating Natural Language Explanations for Visual Question Answering using Scene Graphs and Visual Attention
- Do Explanations make VQA Models more Predictable to a Human?