Preemptively Pruning Clever-Hans Strategies in Deep Neural Networks

Abstract

Robustness has become an important consideration in deep learning. With the help of explainable AI, mismatches between an explained model's decision strategy and the user's domain knowledge (e.g. Clever Hans effects) have been identified as a starting point for improving faulty models. However, it is less clear what to do when the user and the explanation agree. In this paper, we demonstrate that acceptance of explanations by the user is not a guarantee for a machine learning model to be robust against Clever Hans effects, which may remain undetected. Such hidden flaws of the model can nevertheless be mitigated, and we demonstrate this by contributing a new method, Explanation-Guided Exposure Minimization (EGEM), that preemptively prunes variations in the ML model that have not been the subject of positive explanation feedback. Experiments demonstrate that our approach leads to models that strongly reduce their reliance on hidden Clever Hans strategies, and consequently achieve higher accuracy on new data.

arXiv comment18 pages + supplement

References in corpus (25)

• Auto-Encoding Variational Bayes
• Grad-CAM: Visual Explanations from Deep Networks via Gradient-based Localization
• The HAM10000 dataset, a large collection of multi-source dermatoscopic images of common pigmented skin lesions
• On the Opportunities and Risks of Foundation Models
• Interpretable machine learning: definitions, methods, and applications
• Shortcut Learning in Deep Neural Networks
• Explaining Deep Neural Networks and Beyond: A Review of Methods and Applications
• DINOv2: Learning Robust Visual Features without Supervision
• Common pitfalls and recommendations for using machine learning to detect and prognosticate for COVID-19 using chest radiographs and CT scans
• Unmasking Clever Hans Predictors and Assessing What Machines Really Learn
• GANSpace: Discovering Interpretable GAN Controls
• BCN20000: Dermoscopic Lesions in the Wild
• Higher-Order Explanations of Graph Neural Networks via Relevant Walks
• Pruning by Explaining: A Novel Criterion for Deep Neural Network Pruning
• Resolving challenges in deep learning-based analyses of histopathological images using explanation methods
• Toward Explainable AI for Regression Models
• From Clustering to Cluster Explanations via Neural Networks
• Explainable Deep One-Class Classification
• Building and Interpreting Deep Similarity Models
• Testing DNN Image Classifiers for Confusion & Bias Errors
• Software for Dataset-wide XAI: From Local Explanations to Global Insights with Zennit, CoRelAy, and ViRelAy
• Fast On-the-fly Retraining-free Sparsification of Convolutional Neural Networks
• Spread Spurious Attribute: Improving Worst-group Accuracy with Spurious Attribute Estimation
• BARACK: Partially Supervised Group Robustness With Guarantees
• DORA: Exploring Outlier Representations in Deep Neural Networks

Cited by in corpus (2)

• The Clever Hans Effect in Unsupervised Learning
• Disentangled Explanations of Neural Network Predictions by Finding Relevant Subspaces