Explainability Requires Interactivity
arXiv:2109.07869
Abstract
When explaining the decisions of deep neural networks, simple stories are tempting but dangerous. Especially in computer vision, the most popular explanation approaches give a false sense of comprehension to its users and provide an overly simplistic picture. We introduce an interactive framework to understand the highly complex decision boundaries of modern vision models. It allows the user to exhaustively inspect, probe, and test a network's decisions. Across a range of case studies, we compare the power of our interactive approach to static explanation methods, showing how these can lead a user astray, with potentially severe consequences.
References in corpus (4)
- SmoothGrad: removing noise by adding noise
- Captum: A unified and generic model interpretability library for PyTorch
- Towards Realistic Individual Recourse and Actionable Explanations in Black-Box Decision Making Systems
- Using StyleGAN for Visual Interpretability of Deep Learning Models on Medical Images