Interpreting Vision and Language Generative Models with Semantic Visual Priors

Abstract

When applied to Image-to-text models, interpretability methods often provide token-by-token explanations namely, they compute a visual explanation for each token of the generated sequence. Those explanations are expensive to compute and unable to comprehensively explain the model's output. Therefore, these models often require some sort of approximation that eventually leads to misleading explanations. We develop a framework based on SHAP, that allows for generating comprehensive, meaningful explanations leveraging the meaning representation of the output sequence as a whole. Moreover, by exploiting semantic priors in the visual backbone, we extract an arbitrary number of features that allows the efficient computation of Shapley values on large-scale models, generating at the same time highly meaningful visual explanations. We demonstrate that our method generates semantically more expressive explanations than traditional methods at a lower compute cost and that it can be generalized over other explainability methods.

References in corpus (13)

• An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale
• Striving for Simplicity: The All Convolutional Net
• All Models are Wrong, but Many are Useful: Learning a Variable's Importance by Studying an Entire Class of Prediction Models Simultaneously
• Model-Agnostic Interpretability of Machine Learning
• From Anecdotal Evidence to Quantitative Evaluation Methods: A Systematic Review on Evaluating Explainable AI
• Not Just a Black Box: Learning Important Features Through Propagating Activation Differences
• Unsupervised Semantic Segmentation by Distilling Feature Correspondences
• A Comprehensive Review for MRF and CRF Approaches in Pathology Image Analysis
• VALSE: A Task-Independent Benchmark for Vision and Language Models Centered on Linguistic Phenomena
• SEED-Bench: Benchmarking Multimodal LLMs with Generative Comprehension
• Post hoc Explanations may be Ineffective for Detecting Unknown Spurious Correlation
• Fast Hierarchical Games for Image Explanations
• MM-SHAP: A Performance-agnostic Metric for Measuring Multimodal Contributions in Vision and Language Models & Tasks

Cited by in corpus (1)

• MM-SHAP: A Performance-agnostic Metric for Measuring Multimodal Contributions in Vision and Language Models & Tasks