Solving the enigma: Enhancing faithfulness and comprehensibility in explanations of deep networks

Abstract

The accelerated progress of artificial intelligence (AI) has popularized deep learning models across various domains, yet their inherent opacity poses challenges, particularly in critical fields like healthcare, medicine, and the geosciences. Explainable AI (XAI) has emerged to shed light on these 'black box' models, aiding in deciphering their decision-making processes. However, different XAI methods often produce significantly different explanations, leading to high inter-method variability that increases uncertainty and undermines trust in deep networks' predictions. In this study, we address this challenge by introducing a novel framework designed to enhance the explainability of deep networks through a dual focus on maximizing both accuracy and comprehensibility in the explanations. Our framework integrates outputs from multiple established XAI methods and leverages a non-linear neural network model, termed the 'explanation optimizer,' to construct a unified, optimal explanation. The optimizer evaluates explanations using two key metrics: faithfulness (accuracy in reflecting the network's decisions) and complexity (comprehensibility). By balancing these, it provides accurate and accessible explanations, addressing a key XAI limitation. Experiments on multi-class and binary classification in 2D object and 3D neuroscience imaging confirm its efficacy. Our optimizer achieved faithfulness scores 155% and 63% higher than the best XAI methods in 3D and 2D tasks, respectively, while also reducing complexity for better understanding. These results demonstrate that optimal explanations based on specific quality criteria are achievable, offering a solution to the issue of inter-method variability in the current XAI literature and supporting more trustworthy deep network predictions

Accepted manuscript in AI Open Journal (https://www.sciencedirect.com/journal/ai-open)

References in corpus (7)

• An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale
• Striving for Simplicity: The All Convolutional Net
• Explaining Deep Neural Networks and Beyond: A Review of Methods and Applications
• Explainable Artificial Intelligence (XAI) 2.0: A Manifesto of Open Challenges and Interdisciplinary Research Directions
• Quantus: An Explainable AI Toolkit for Responsible Evaluation of Neural Network Explanations and Beyond
• Neural Network Attribution Methods for Problems in Geoscience: A Novel Synthetic Benchmark Dataset
• Investigating the fidelity of explainable artificial intelligence methods for applications of convolutional neural networks in geoscience