Software for Dataset-wide XAI: From Local Explanations to Global Insights with Zennit, CoRelAy, and ViRelAy

Abstract

Deep Neural Networks (DNNs) are known to be strong predictors, but their prediction strategies can rarely be understood. With recent advances in Explainable Artificial Intelligence (XAI), approaches are available to explore the reasoning behind those complex models' predictions. Among post-hoc attribution methods, Layer-wise Relevance Propagation (LRP) shows high performance. For deeper quantitative analysis, manual approaches exist, but without the right tools they are unnecessarily labor intensive. In this software paper, we introduce three software packages targeted at scientists to explore model reasoning using attribution approaches and beyond: (1) Zennit - a highly customizable and intuitive attribution framework implementing LRP and related approaches in PyTorch, (2) CoRelAy - a framework to easily and quickly construct quantitative analysis pipelines for dataset-wide analyses of explanations, and (3) ViRelAy - a web-application to interactively explore data, attributions, and analysis results. With this, we provide a standardized implementation solution for XAI, to contribute towards more reproducibility in our field.

20 pages, 6 figures, 2 listings, 1 table

Cited by in corpus (8)

• From Attribution Maps to Human-Understandable Explanations through Concept Relevance Propagation
• Toward Explainable AI for Regression Models
• Explaining Deep Learning for ECG Analysis: Building Blocks for Auditing and Knowledge Discovery
• Disentangled Explanations of Neural Network Predictions by Finding Relevant Subspaces
• Human-Centered Evaluation of XAI Methods
• Preemptively Pruning Clever-Hans Strategies in Deep Neural Networks
• Robustness of Visual Explanations to Common Data Augmentation
• Bridging the Gap: Gaze Events as Interpretable Concepts to Explain Deep Neural Sequence Models