Explainable Artificial Intelligence (XAI) 2.0: A Manifesto of Open Challenges and Interdisciplinary Research Directions

Abstract

As systems based on opaque Artificial Intelligence (AI) continue to flourish in diverse real-world applications, understanding these black box models has become paramount. In response, Explainable AI (XAI) has emerged as a field of research with practical and ethical benefits across various domains. This paper not only highlights the advancements in XAI and its application in real-world scenarios but also addresses the ongoing challenges within XAI, emphasizing the need for broader perspectives and collaborative efforts. We bring together experts from diverse fields to identify open problems, striving to synchronize research agendas and accelerate XAI in practical applications. By fostering collaborative discussion and interdisciplinary cooperation, we aim to propel XAI forward, contributing to its continued success. Our goal is to put forward a comprehensive proposal for advancing XAI. To achieve this goal, we present a manifesto of 27 open problems categorized into nine categories. These challenges encapsulate the complexities and nuances of XAI and offer a road map for future research. For each problem, we provide promising research directions in the hope of harnessing the collective intelligence of interested stakeholders.

References in corpus (53)

• Towards A Rigorous Science of Interpretable Machine Learning
• A Survey on Explainable Artificial Intelligence (XAI): Towards Medical XAI
• Interpretable machine learning: definitions, methods, and applications
• Shortcut Learning in Deep Neural Networks
• Diffusion Models in Vision: A Survey
• Explaining Deep Neural Networks and Beyond: A Review of Methods and Applications
• Unmasking Clever Hans Predictors and Assessing What Machines Really Learn
• The role of explainability in creating trustworthy artificial intelligence for health care: a comprehensive survey of the terminology, design choices, and evaluation strategies
• Explaining Explanations in AI
• What Do We Want From Explainable Artificial Intelligence (XAI)? -- A Stakeholder Perspective on XAI and a Conceptual Model Guiding Interdisciplinary XAI Research
• This Looks Like That: Deep Learning for Interpretable Image Recognition
• From Anecdotal Evidence to Quantitative Evaluation Methods: A Systematic Review on Evaluating Explainable AI
• A Comprehensive Taxonomy for Explainable Artificial Intelligence: A Systematic Survey of Surveys on Methods and Concepts
• Survey on Federated Learning Threats: concepts, taxonomy on attacks and defences, experimental study and challenges
• One Explanation Does Not Fit All: A Toolkit and Taxonomy of AI Explainability Techniques
• Pruning by Explaining: A Novel Criterion for Deep Neural Network Pruning
• The Pragmatic Turn in Explainable Artificial Intelligence (XAI)
• TabTransformer: Tabular Data Modeling Using Contextual Embeddings
• Ground Truth Evaluation of Neural Network Explanations with CLEVR-XAI
• From Attribution Maps to Human-Understandable Explanations through Concept Relevance Propagation
• On the Explainability of Natural Language Processing Deep Models
• Explainable Artificial Intelligence (XAI) on TimeSeries Data: A Survey
• Robust Explainability: A Tutorial on Gradient-Based Attribution Methods for Deep Neural Networks
• A Systematic Review on Model Watermarking for Neural Networks
• OpenXAI: Towards a Transparent Evaluation of Model Explanations
• ProtoPShare: Prototype Sharing for Interpretable Image Classification and Similarity Discovery
• Is Neuro-Symbolic AI Meeting its Promise in Natural Language Processing? A Structured Review
• On Cognitive Preferences and the Plausibility of Rule-based Models
• A Survey of Machine Unlearning
• How model accuracy and explanation fidelity influence user trust
• Progress measures for grokking via mechanistic interpretability
• CLOSURE: Assessing Systematic Generalization of CLEVR Models
• Kandinsky Patterns
• Which Explanation Should I Choose? A Function Approximation Perspective to Characterizing Post Hoc Explanations
• Sparse Visual Counterfactual Explanations in Image Space
• Impossibility Results in AI: A Survey
• AtMan: Understanding Transformer Predictions Through Memory Efficient Attention Manipulation
• Explainability in reinforcement learning: perspective and position
• Toward Transparent AI: A Survey on Interpreting the Inner Structures of Deep Neural Networks
• Bridging the Gap Between Explainable AI and Uncertainty Quantification to Enhance Trustability
• Reflective-Net: Learning from Explanations
• The Clock and the Pizza: Two Stories in Mechanistic Explanation of Neural Networks
• AttnLRP: Attention-Aware Layer-Wise Relevance Propagation for Transformers
• Vertical Federated Learning: A Structured Literature Review
• REVEL Framework to measure Local Linear Explanations for black-box models: Deep Learning Image Classification case of study
• Going Beyond XAI: A Systematic Survey for Explanation-Guided Learning
• Scale Alone Does not Improve Mechanistic Interpretability in Vision Models
• From Robustness to Explainability and Back Again
• Interpreting Neural Networks through the Polytope Lens
• Explainability is NOT a Game
• Can Transformers Learn to Solve Problems Recursively?
• A Study of Compositional Generalization in Neural Models
• Counterfactual Explanations of Concept Drift

Cited by in corpus (25)

• Explainable Artificial Intelligence: A Survey of Needs, Techniques, Applications, and Future Direction
• Interpretable Clustering: A Survey
• Describe, Transform, Machine Learning: Feature Engineering for Grain Boundaries and Other Variable-Sized Atom Clusters
• What Does Evaluation of Explainable Artificial Intelligence Actually Tell Us? A Case for Compositional and Contextual Validation of XAI Building Blocks
• Benchmarking Instance-Centric Counterfactual Algorithms for XAI: From White Box to Black Box
• An Open API Architecture to Discover the Trustworthy Explanation of Cloud AI Services
• Rad4XCNN: a new agnostic method for post-hoc global explanation of CNN-derived features by means of radiomics
• VR-FuseNet: A Fusion of Heterogeneous Fundus Data and Explainable Deep Network for Diabetic Retinopathy Classification
• LIMEtree: Consistent and Faithful Surrogate Explanations of Multiple Classes
• GPT Assisted Annotation of Rhetorical and Linguistic Features for Interpretable Propaganda Technique Detection in News Text
• Euclid preparation. LXVIII. Extracting physical parameters from galaxies with machine learning
• "Oh LLM, I'm Asking Thee, Please Give Me a Decision Tree": Zero-Shot Decision Tree Induction and Embedding with Large Language Models
• Solving the enigma: Enhancing faithfulness and comprehensibility in explanations of deep networks
• CiteFusion: An Ensemble Framework for Citation Intent Classification Harnessing Dual-Model Binary Couples and SHAP Analyses
• Design Considerations for Human Oversight of AI: Insights from Co-Design Workshops and Work Design Theory
• FovEx: Human-Inspired Explanations for Vision Transformers and Convolutional Neural Networks
• Give Me a Choice: The Consequences of Restricting Choices Through AI-Support for Perceived Autonomy, Motivational Variables, and Decision Performance
• Local Concept Embeddings for Analysis of Concept Distributions in Vision DNN Feature Spaces
• Time Series Information Visualization -- A Review of Approaches and Tools
• Global Counterfactual Directions
• Genetic Programming for Explainable Manifold Learning
• M3Net: A Macro-to-Meso-to-Micro Clinical-inspired Hierarchical 3D Network for Pulmonary Nodule Classification
• Knowledge graphs for empirical concept retrieval
• X-SHIELD: Regularization for eXplainable Artificial Intelligence
• Explainability of Algorithms