The Thousand Faces of Explainable AI Along the Machine Learning Life Cycle: Industrial Reality and Current State of Research
arXiv:2310.07882 · doi:10.1007/978-3-031-35891-3_13
Abstract
In this paper, we investigate the practical relevance of explainable artificial intelligence (XAI) with a special focus on the producing industries and relate them to the current state of academic XAI research. Our findings are based on an extensive series of interviews regarding the role and applicability of XAI along the Machine Learning (ML) lifecycle in current industrial practice and its expected relevance in the future. The interviews were conducted among a great variety of roles and key stakeholders from different industry sectors. On top of that, we outline the state of XAI research by providing a concise review of the relevant literature. This enables us to provide an encompassing overview covering the opinions of the surveyed persons as well as the current state of academic research. By comparing our interview results with the current research approaches we reveal several discrepancies. While a multitude of different XAI approaches exists, most of them are centered around the model evaluation phase and data scientists. Their versatile capabilities for other stages are currently either not sufficiently explored or not popular among practitioners. In line with existing work, our findings also confirm that more efforts are needed to enable also non-expert users' interpretation and understanding of opaque AI models with existing methods and frameworks.
arXiv commentInternational Conference on Human-Computer Interaction 2023
References in corpus (36)
- Explaining and Harnessing Adversarial Examples
- Towards A Rigorous Science of Interpretable Machine Learning
- One pixel attack for fooling deep neural networks
- Understanding Black-box Predictions via Influence Functions
- Improving fairness in machine learning systems: What do industry practitioners need?
- Captum: A unified and generic model interpretability library for PyTorch
- Attention in Natural Language Processing
- Opportunities and Challenges in Explainable Artificial Intelligence (XAI): A Survey
- Interpretability Beyond Feature Attribution: Quantitative Testing with Concept Activation Vectors (TCAV)
- InterpretML: A Unified Framework for Machine Learning Interpretability
- Pruning by Explaining: A Novel Criterion for Deep Neural Network Pruning
- Towards Efficient Data Valuation Based on the Shapley Value
- Estimating Training Data Influence by Tracing Gradient Descent
- Explainable Artificial Intelligence (XAI) on TimeSeries Data: A Survey
- What Neural Networks Memorize and Why: Discovering the Long Tail via Influence Estimation
- The US Algorithmic Accountability Act of 2022 vs. The EU Artificial Intelligence Act: What can they learn from each other?
- On the Accuracy of Influence Functions for Measuring Group Effects
- Benchmarking and Survey of Explanation Methods for Black Box Models
- Fast TreeSHAP: Accelerating SHAP Value Computation for Trees
- Input Similarity from the Neural Network Perspective
- A Distributional Framework for Data Valuation
- The Brussels Effect and Artificial Intelligence: How EU regulation will impact the global AI market
- Data Cleansing for Models Trained with SGD
- The Shapley Value of Tuples in Query Answering
- Evaluation of Similarity-based Explanations
- Efficient computation and analysis of distributional Shapley values
- Learning to Estimate Shapley Values with Vision Transformers
- If Influence Functions are the Answer, Then What is the Question?
- A Psychological Theory of Explainability
- Human-interpretable model explainability on high-dimensional data
- A Unified Framework for Task-Driven Data Quality Management
- FROTE: Feedback Rule-Driven Oversampling for Editing Models
- Accelerating Shapley Explanation via Contributive Cooperator Selection
- Improving Cooperative Game Theory-based Data Valuation via Data Utility Learning
- Explanation Shift: Detecting distribution shifts on tabular data via the explanation space
- Linear TreeShap