Investigating and Mitigating Degree-Related Biases in Graph Convolutional Networks

Abstract

Graph Convolutional Networks (GCNs) show promising results for semi-supervised learning tasks on graphs, thus become favorable comparing with other approaches. Despite the remarkable success of GCNs, it is difficult to train GCNs with insufficient supervision. When labeled data are limited, the performance of GCNs becomes unsatisfying for low-degree nodes. While some prior work analyze successes and failures of GCNs on the entire model level, profiling GCNs on individual node level is still underexplored. In this paper, we analyze GCNs in regard to the node degree distribution. From empirical observation to theoretical proof, we confirm that GCNs are biased towards nodes with larger degrees with higher accuracy on them, even if high-degree nodes are underrepresented in most graphs. We further develop a novel Self-Supervised-Learning Degree-Specific GCN (SL-DSGC) that mitigate the degree-related biases of GCNs from model and data aspects. Firstly, we propose a degree-specific GCN layer that captures both discrepancies and similarities of nodes with different degrees, which reduces the inner model-aspect biases of GCNs caused by sharing the same parameters with all nodes. Secondly, we design a self-supervised-learning algorithm that creates pseudo labels with uncertainty scores on unlabeled nodes with a Bayesian neural network. Pseudo labels increase the chance of connecting to labeled neighbors for low-degree nodes, thus reducing the biases of GCNs from the data perspective. Uncertainty scores are further exploited to weight pseudo labels dynamically in the stochastic gradient descent for SL-DSGC. Experiments on three benchmark datasets show SL-DSGC not only outperforms state-of-the-art self-training/self-supervised-learning GCN methods, but also improves GCN accuracy dramatically for low-degree nodes.

arXiv commentAccepted to CIKM 2020

References in corpus (4)

• A Comprehensive Survey on Graph Neural Networks
• Graph Convolutional Neural Networks for Web-Scale Recommender Systems
• A Semi-supervised Graph Attentive Network for Financial Fraud Detection
• Transferring Robustness for Graph Neural Network Against Poisoning Attacks

Cited by in corpus (15)

• Self-supervised Graph Learning for Recommendation
• Improving Fairness in Graph Neural Networks via Mitigating Sensitive Attribute Leakage
• A Comprehensive Survey on Trustworthy Graph Neural Networks: Privacy, Robustness, Fairness, and Explainability
• Should Graph Convolution Trust Neighbors? A Simple Causal Inference Method
• LTE4G: Long-Tail Experts for Graph Neural Networks
• RawlsGCN: Towards Rawlsian Difference Principle on Graph Convolutional Network
• Toward Degree Bias in Embedding-Based Knowledge Graph Completion
• Relational Self-Supervised Learning on Graphs
• GraFN: Semi-Supervised Node Classification on Graph with Few Labels via Non-Parametric Distribution Assignment
• Pitfalls in Link Prediction with Graph Neural Networks: Understanding the Impact of Target-link Inclusion & Better Practices
• Causality and Independence Enhancement for Biased Node Classification
• FairSample: Training Fair and Accurate Graph Convolutional Neural Networks Efficiently
• A Comprehensive Benchmark on Spectral GNNs: The Impact on Efficiency, Memory, and Effectiveness
• Causality-aware Graph Aggregation Weight Estimator for Popularity Debiasing in Top-K Recommendation
• Implications of Topological Imbalance for Representation Learning on Biomedical Knowledge Graphs