ProtoShotXAI: Using Prototypical Few-Shot Architecture for Explainable AI

Abstract

Unexplainable black-box models create scenarios where anomalies cause deleterious responses, thus creating unacceptable risks. These risks have motivated the field of eXplainable Artificial Intelligence (XAI) to improve trust by evaluating local interpretability in black-box neural networks. Unfortunately, the ground truth is unavailable for the model's decision, so evaluation is limited to qualitative assessment. Further, interpretability may lead to inaccurate conclusions about the model or a false sense of trust. We propose to improve XAI from the vantage point of the user's trust by exploring a black-box model's latent feature space. We present an approach, ProtoShotXAI, that uses a Prototypical few-shot network to explore the contrastive manifold between nonlinear features of different classes. A user explores the manifold by perturbing the input features of a query sample and recording the response for a subset of exemplars from any class. Our approach is the first locally interpretable XAI model that can be extended to, and demonstrated on, few-shot networks. We compare ProtoShotXAI to the state-of-the-art XAI approaches on MNIST, Omniglot, and ImageNet to demonstrate, both quantitatively and qualitatively, that ProtoShotXAI provides more flexibility for model exploration. Finally, ProtoShotXAI also demonstrates novel explainabilty and detectabilty on adversarial samples.

arXiv comment38 pages, 13 figures, 5 tables

References in corpus (13)

• Prototypical Networks for Few-shot Learning
• Grad-CAM++: Improved Visual Explanations for Deep Convolutional Networks
• Axiomatic Attribution for Deep Networks
• Understanding Black-box Predictions via Influence Functions
• Data Augmentation Generative Adversarial Networks
• SmoothGrad: removing noise by adding noise
• ImageNet-trained CNNs are biased towards texture; increasing shape bias improves accuracy and robustness
• Deep k-Nearest Neighbors: Towards Confident, Interpretable and Robust Deep Learning
• Prototype selection for interpretable classification
• Adapted Deep Embeddings: A Synthesis of Methods for $k$-Shot Inductive Transfer Learning
• Deep Triplet Ranking Networks for One-Shot Recognition
• Local Interpretable Model-agnostic Explanations of Bayesian Predictive Models via Kullback-Leibler Projections
• The Bouncer Problem: Challenges to Remote Explainability