XAlgo: a Design Probe of Explaining Algorithms' Internal States via Question-Answering
arXiv:2007.07407
Abstract
Algorithms often appear as 'black boxes' to non-expert users. While prior work focuses on explainable representations and expert-oriented exploration, we propose and study an interactive approach using question answering to explain deterministic algorithms to non-expert users who need to understand the algorithms' internal states (e.g., students learning algorithms, operators monitoring robots, admins troubleshooting network routing). We construct XAlgo -- a formal model that first classifies the type of question based on a taxonomy and generates an answer based on a set of rules that extract information from representations of an algorithm's internal states, e.g., the pseudocode. A design probe in an algorithm learning scenario with 18 participants (9 for a Wizard-of-Oz XAlgo and 9 as a control group) reports findings and design implications based on what kinds of questions people ask, how well XAlgo responds, and what remain as challenges to bridge users' gulf of understanding algorithms.
References in corpus (10)
- Learning Important Features Through Propagating Activation Differences
- VQA: Visual Question Answering
- SQuAD: 100,000+ Questions for Machine Comprehension of Text
- Visualizing Deep Neural Network Decisions: Prediction Difference Analysis
- What Does Explainable AI Really Mean? A New Conceptualization of Perspectives
- SQLNet: Generating Structured Queries From Natural Language Without Reinforcement Learning
- Distilling a Neural Network Into a Soft Decision Tree
- Metrics for Explainable AI: Challenges and Prospects
- Contrastive Explanation: A Structural-Model Approach
- LSTMVis: A Tool for Visual Analysis of Hidden State Dynamics in Recurrent Neural Networks