Proactive Pseudo-Intervention: Causally Informed Contrastive Learning For Interpretable Vision Models

Abstract

Deep neural networks excel at comprehending complex visual signals, delivering on par or even superior performance to that of human experts. However, ad-hoc visual explanations of model decisions often reveal an alarming level of reliance on exploiting non-causal visual cues that strongly correlate with the target label in training data. As such, deep neural nets suffer compromised generalization to novel inputs collected from different sources, and the reverse engineering of their decision rules offers limited interpretability. To overcome these limitations, we present a novel contrastive learning strategy called {\it Proactive Pseudo-Intervention} (PPI) that leverages proactive interventions to guard against image features with no causal relevance. We also devise a novel causally informed salience mapping module to identify key image pixels to intervene, and show it greatly facilitates model interpretability. To demonstrate the utility of our proposals, we benchmark on both standard natural images and challenging medical image datasets. PPI-enhanced models consistently deliver superior performance relative to competing solutions, especially on out-of-domain predictions and data integration from heterogeneous sources. Further, our causally trained saliency maps are more succinct and meaningful relative to their non-causal counterparts.

References in corpus (17)

• Very Deep Convolutional Networks for Large-Scale Image Recognition
• A Simple Framework for Contrastive Learning of Visual Representations
• Grad-CAM++: Improved Visual Explanations for Deep Convolutional Networks
• Learning Important Features Through Propagating Activation Differences
• Deep Inside Convolutional Networks: Visualising Image Classification Models and Saliency Maps
• Sanity Checks for Saliency Maps
• Contrastive Multiview Coding
• A Benchmark for Interpretability Methods in Deep Neural Networks
• Invariant Risk Minimization
• Causality for Machine Learning
• Supervised Contrastive Learning
• Learning Robust Representations by Projecting Superficial Statistics Out
• Explaining Image Classifiers by Counterfactual Generation
• TensorNetwork for Machine Learning
• A Causal View on Robustness of Neural Networks
• Regional Multi-scale Approach for Visually Pleasing Explanations of Deep Neural Networks
• Learning Causal Structures Using Regression Invariance