Causal Attention for Interpretable and Generalizable Graph Classification

Abstract

In graph classification, attention and pooling-based graph neural networks (GNNs) prevail to extract the critical features from the input graph and support the prediction. They mostly follow the paradigm of learning to attend, which maximizes the mutual information between the attended graph and the ground-truth label. However, this paradigm makes GNN classifiers recklessly absorb all the statistical correlations between input features and labels in the training data, without distinguishing the causal and noncausal effects of features. Instead of underscoring the causal features, the attended graphs are prone to visit the noncausal features as the shortcut to predictions. Such shortcut features might easily change outside the training distribution, thereby making the GNN classifiers suffer from poor generalization. In this work, we take a causal look at the GNN modeling for graph classification. With our causal assumption, the shortcut feature serves as a confounder between the causal feature and prediction. It tricks the classifier to learn spurious correlations that facilitate the prediction in in-distribution (ID) test evaluation, while causing the performance drop in out-of-distribution (OOD) test data. To endow the classifier with better interpretation and generalization, we propose the Causal Attention Learning (CAL) strategy, which discovers the causal patterns and mitigates the confounding effect of shortcuts. Specifically, we employ attention modules to estimate the causal and shortcut features of the input graph. We then parameterize the backdoor adjustment of causal theory -- combine each causal feature with various shortcut features. It encourages the stable relationships between the causal estimation and prediction, regardless of the changes in shortcut parts and distributions. Extensive experiments on synthetic and real-world datasets demonstrate the effectiveness of CAL.

Accepted to KDD 2022

References in corpus (9)

• An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale
• Shortcut Learning in Deep Neural Networks
• Self-Attention Graph Pooling
• TUDataset: A collection of benchmark datasets for learning with graphs
• How to Find Your Friendly Neighborhood: Graph Attention Design with Self-Supervision
• The Risks of Invariant Risk Minimization
• Should Graph Convolution Trust Neighbors? A Simple Causal Inference Method
• Discovering Invariant Rationales for Graph Neural Networks
• Reinforced Causal Explainer for Graph Neural Networks

Cited by in corpus (11)

• Graph Neural Networks for Financial Fraud Detection: A Review
• Causality-based CTR Prediction using Graph Neural Networks
• Towards Fair Graph Neural Networks via Graph Counterfactual
• Causal invariant geographic network representations with feature and structural distribution shifts
• GraphGLOW: Universal and Generalizable Structure Learning for Graph Neural Networks
• Counterfactual Learning on Graphs: A Survey
• Causality-Aware Spatiotemporal Graph Neural Networks for Spatiotemporal Time Series Imputation
• Unleashing the Power of Large Language Model for Denoising Recommendation
• Causality and Independence Enhancement for Biased Node Classification
• Predicting Scientific Impact Through Diffusion, Conformity, and Contribution Disentanglement
• Toward a benchmark for CTR prediction in online advertising: datasets, evaluation protocols and perspectives