Explainability in AI Based Applications: A Framework for Comparing Different Techniques

Abstract

The integration of artificial intelligence into business processes has significantly enhanced decision-making capabilities across various industries such as finance, healthcare, and retail. However, explaining the decisions made by these AI systems poses a significant challenge due to the opaque nature of recent deep learning models, which typically function as black boxes. To address this opacity, a multitude of explainability techniques have emerged. However, in practical business applications, the challenge lies in selecting an appropriate explainability method that balances comprehensibility with accuracy. This paper addresses the practical need of understanding differences in the output of explainability techniques by proposing a novel method for the assessment of the agreement of different explainability techniques. Based on our proposed methods, we provide a comprehensive comparative analysis of six leading explainability techniques to help guiding the selection of such techniques in practice. Our proposed general-purpose method is evaluated on top of one of the most popular deep learning architectures, the Vision Transformer model, which is frequently employed in business applications. Notably, we propose a novel metric to measure the agreement of explainability techniques that can be interpreted visually. By providing a practical framework for understanding the agreement of diverse explainability techniques, our research aims to facilitate the broader integration of interpretable AI systems in business applications.

10 pages, 5 figures

References in corpus (12)

• An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale
• Towards A Rigorous Science of Interpretable Machine Learning
• European Union regulations on algorithmic decision-making and a "right to explanation"
• Explaining Deep Neural Networks and Beyond: A Review of Methods and Applications
• Explaining Explanations in AI
• Model-Agnostic Interpretability of Machine Learning
• What do we need to build explainable AI systems for the medical domain?
• Attention is not Explanation
• Fooling LIME and SHAP: Adversarial Attacks on Post hoc Explanation Methods
• Robust Explainability: A Tutorial on Gradient-Based Attribution Methods for Deep Neural Networks
• Explainable Artificial Intelligence for Autonomous Driving: A Comprehensive Overview and Field Guide for Future Research Directions
• Interpretability-Aware Vision Transformer