Feature Overcorrelation in Deep Graph Neural Networks: A New Perspective
arXiv:2206.07743 · doi:10.1145/3534678.3539445
Abstract
Recent years have witnessed remarkable success achieved by graph neural networks (GNNs) in many real-world applications such as recommendation and drug discovery. Despite the success, oversmoothing has been identified as one of the key issues which limit the performance of deep GNNs. It indicates that the learned node representations are highly indistinguishable due to the stacked aggregators. In this paper, we propose a new perspective to look at the performance degradation of deep GNNs, i.e., feature overcorrelation. Through empirical and theoretical study on this matter, we demonstrate the existence of feature overcorrelation in deeper GNNs and reveal potential reasons leading to this issue. To reduce the feature correlation, we propose a general framework DeCorr which can encourage GNNs to encode less redundant information. Extensive experiments have demonstrated that DeCorr can help enable deeper GNNs and is complementary to existing techniques tackling the oversmoothing issue.
Accepted by KDD 2022
References in corpus (12)
- Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift
- Deep Convolutional Networks on Graph-Structured Data
- Towards Deeper Graph Neural Networks
- Simple and Deep Graph Convolutional Networks
- Few-Shot Graph Learning for Molecular Property Prediction
- Graph Contrastive Learning Automated
- Improving Fairness in Graph Neural Networks via Mitigating Sensitive Attribute Leakage
- Towards Deeper Graph Neural Networks with Differentiable Group Normalization
- Graph Neural Networks for Multimodal Single-Cell Data Integration
- Graph Data Augmentation for Graph Machine Learning: A Survey
- Data Augmentation for Deep Graph Learning: A Survey
- Automated Self-Supervised Learning for Graphs