Global Counterfactual Directions
arXiv:2404.12488 · doi:10.1007/978-3-031-73036-8_5
Abstract
Despite increasing progress in development of methods for generating visual counterfactual explanations, especially with the recent rise of Denoising Diffusion Probabilistic Models, previous works consider them as an entirely local technique. In this work, we take the first step at globalizing them. Specifically, we discover that the latent space of Diffusion Autoencoders encodes the inference process of a given classifier in the form of global directions. We propose a novel proxy-based approach that discovers two types of these directions with the use of only single image in an entirely black-box manner. Precisely, g-directions allow for flipping the decision of a given classifier on an entire dataset of images, while h-directions further increase the diversity of explanations. We refer to them in general as Global Counterfactual Directions (GCDs). Moreover, we show that GCDs can be naturally combined with Latent Integrated Gradients resulting in a new black-box attribution method, while simultaneously enhancing the understanding of counterfactual explanations. We validate our approach on existing benchmarks and show that it generalizes to real-world use-cases.
ECCV 2024
References in corpus (9)
- Explaining Machine Learning Classifiers through Diverse Counterfactual Explanations
- Segment Anything Model for Medical Image Analysis: an Experimental Study
- Explainable Artificial Intelligence (XAI) 2.0: A Manifesto of Open Challenges and Interdisciplinary Research Directions
- Diffusion Models already have a Semantic Latent Space
- Sparse Visual Counterfactual Explanations in Image Space
- Understanding the Latent Space of Diffusion Models through the Lens of Riemannian Geometry
- Generating Interpretable Counterfactual Explanations By Implicit Minimisation of Epistemic and Aleatoric Uncertainties
- Bridging the Sim2Real gap with CARE: Supervised Detection Adaptation with Conditional Alignment and Reweighting
- LANCE: Stress-testing Visual Models by Generating Language-guided Counterfactual Images