explAIner: A Visual Analytics Framework for Interactive and Explainable Machine Learning

Abstract

We propose a framework for interactive and explainable machine learning that enables users to (1) understand machine learning models; (2) diagnose model limitations using different explainable AI methods; as well as (3) refine and optimize the models. Our framework combines an iterative XAI pipeline with eight global monitoring and steering mechanisms, including quality monitoring, provenance tracking, model comparison, and trust building. To operationalize the framework, we present explAIner, a visual analytics system for interactive and explainable machine learning that instantiates all phases of the suggested pipeline within the commonly used TensorBoard environment. We performed a user-study with nine participants across different expertise levels to examine their perception of our workflow and to collect suggestions to fill the gap between our system and framework. The evaluation confirms that our tightly integrated system leads to an informed machine learning process while disclosing opportunities for further extensions.

arXiv comment9 pages paper, 2 pages references, 5 pages supplementary material (ancillary files)

References in corpus (19)

• Evaluation: from precision, recall and F-measure to ROC, informedness, markedness and correlation
• Towards A Rigorous Science of Interpretable Machine Learning
• Axiomatic Attribution for Deep Networks
• Learning Important Features Through Propagating Activation Differences
• Neural Architecture Search: A Survey
• Explaining NonLinear Classification Decisions with Deep Taylor Decomposition
• Recent Trends in Deep Learning Based Natural Language Processing
• SmoothGrad: removing noise by adding noise
• Not Just a Black Box: Learning Important Features Through Propagating Activation Differences
• Deep Reinforcement Learning that Matters
• RetainVis: Visual Analytics with Interpretable and Interactive Recurrent Neural Networks on Electronic Medical Records
• Manifold: A Model-Agnostic Framework for Interpretation and Diagnosis of Machine Learning Models
• GAN Lab: Understanding Complex Deep Generative Models using Interactive Visual Experimentation
• Do Convolutional Neural Networks Learn Class Hierarchy?
• The State of the Art in Integrating Machine Learning into Visual Analytics
• Direct-Manipulation Visualization of Deep Networks
• Machine Teaching: A New Paradigm for Building Machine Learning Systems
• iNNvestigate neural networks!
• Analyzing the Noise Robustness of Deep Neural Networks

Cited by in corpus (41)

• Expanding Explainability: Towards Social Transparency in AI systems
• A Comprehensive Taxonomy for Explainable Artificial Intelligence: A Systematic Survey of Surveys on Methods and Concepts
• The State of the Art in Enhancing Trust in Machine Learning Models with the Use of Visualizations
• A Survey on Explainable Artificial Intelligence for Cybersecurity
• A Survey on ML4VIS: Applying Machine Learning Advances to Data Visualization
• A Survey on AI Assurance
• Explainable Artificial Intelligence: a Systematic Review
• Directive Explanations for Monitoring the Risk of Diabetes Onset: Introducing Directive Data-Centric Explanations and Combinations to Support What-If Explorations
• Is Conversational XAI All You Need? Human-AI Decision Making With a Conversational XAI Assistant
• The Grammar of Interactive Explanatory Model Analysis
• EXMOS: Explanatory Model Steering Through Multifaceted Explanations and Data Configurations
• RELIC: Investigating Large Language Model Responses using Self-Consistency
• Learning CNN filters from user-drawn image markers for coconut-tree image classification
• HyperTendril: Visual Analytics for User-Driven Hyperparameter Optimization of Deep Neural Networks
• Improving Visualization Interpretation Using Counterfactuals
• Angler: Helping Machine Translation Practitioners Prioritize Model Improvements
• XAutoML: A Visual Analytics Tool for Understanding and Validating Automated Machine Learning
• Explainable Predictive Process Monitoring: A User Evaluation
• A Survey of Visual Analytics Techniques for Machine Learning
• Towards Directive Explanations: Crafting Explainable AI Systems for Actionable Human-AI Interactions
• Explainable cognitive decline detection in free dialogues with a Machine Learning approach based on pre-trained Large Language Models
• Should We Trust (X)AI? Design Dimensions for Structured Experimental Evaluations
• DECE: Decision Explorer with Counterfactual Explanations for Machine Learning Models
• A Heuristic Approach for Dual Expert/End-User Evaluation of Guidance in Visual Analytics
• Detecting and explaining postpartum depression in real-time with generative artificial intelligence
• JailbreakLens: Visual Analysis of Jailbreak Attacks Against Large Language Models
• Representation Debiasing of Generated Data Involving Domain Experts
• "Is It My Turn?" Assessing Teamwork and Taskwork in Collaborative Immersive Analytics
• Simulation-Based Optimization of User Interfaces for Quality-Assuring Machine Learning Model Predictions
• Developing Future Human-Centered Smart Cities: Critical Analysis of Smart City Security, Interpretability, and Ethical Challenges
• generAItor: Tree-in-the-Loop Text Generation for Language Model Explainability and Adaptation
• AI-Spectra: A Visual Dashboard for Model Multiplicity to Enhance Informed and Transparent Decision-Making
• An Empirical Study of Explainable AI Techniques on Deep Learning Models For Time Series Tasks
• PipelineProfiler: A Visual Analytics Tool for the Exploration of AutoML Pipelines
• Explainable e-sports win prediction through Machine Learning classification in streaming
• PSEUDo: Interactive Pattern Search in Multivariate Time Series with Locality-Sensitive Hashing and Relevance Feedback
• A Visual Analytics Framework for Explaining and Diagnosing Transfer Learning Processes
• Explainable Machine Learning: The Importance of a System-Centric Perspective
• Communication Analysis through Visual Analytics: Current Practices, Challenges, and New Frontiers
• Towards the Visualization of Aggregated Class Activation Maps to Analyse the Global Contribution of Class Features
• Practitioner Motives to Use Different Hyperparameter Optimization Methods