Quantus: An Explainable AI Toolkit for Responsible Evaluation of Neural Network Explanations and Beyond

Abstract

The evaluation of explanation methods is a research topic that has not yet been explored deeply, however, since explainability is supposed to strengthen trust in artificial intelligence, it is necessary to systematically review and compare explanation methods in order to confirm their correctness. Until now, no tool with focus on XAI evaluation exists that exhaustively and speedily allows researchers to evaluate the performance of explanations of neural network predictions. To increase transparency and reproducibility in the field, we therefore built Quantus -- a comprehensive, evaluation toolkit in Python that includes a growing, well-organised collection of evaluation metrics and tutorials for evaluating explainable methods. The toolkit has been thoroughly tested and is available under an open-source license on PyPi (or on https://github.com/understandable-machine-intelligence-lab/Quantus/).

4 pages, 1 figure, 1 table

Cited by in corpus (19)

• Explainable Artificial Intelligence: A Survey of Needs, Techniques, Applications, and Future Direction
• Adversarial attacks and defenses in explainable artificial intelligence: A survey
• Opening the Black-Box: A Systematic Review on Explainable AI in Remote Sensing
• Towards Evaluating Explanations of Vision Transformers for Medical Imaging
• On the Black-box Explainability of Object Detection Models for Safe and Trustworthy Industrial Applications
• SoK: Modeling Explainability in Security Analytics for Interpretability, Trustworthiness, and Usability
• Human-Centered Evaluation of XAI Methods
• Classification Metrics for Image Explanations: Towards Building Reliable XAI-Evaluations
• Saliency-Bench: A Comprehensive Benchmark for Evaluating Visual Explanations
• Robustness of Visual Explanations to Common Data Augmentation
• Bridging the Gap: Gaze Events as Interpretable Concepts to Explain Deep Neural Sequence Models
• Solving the enigma: Enhancing faithfulness and comprehensibility in explanations of deep networks
• Quantitative Analysis of Primary Attribution Explainable Artificial Intelligence Methods for Remote Sensing Image Classification
• On the Robustness of Global Feature Effect Explanations
• Beyond the Veil of Similarity: Quantifying Semantic Continuity in Explainable AI
• Sparse Explanations of Neural Networks Using Pruned Layer-Wise Relevance Propagation
• LaFAM: Unsupervised Feature Attribution with Label-free Activation Maps
• Meta-evaluating stability measures: MAX-Senstivity & AVG-Sensitivity
• Explanatory Model Monitoring to Understand the Effects of Feature Shifts on Performance