Distribution Consistency based Self-Training for Graph Neural Networks with Sparse Labels

Abstract

Few-shot node classification poses a significant challenge for Graph Neural Networks (GNNs) due to insufficient supervision and potential distribution shifts between labeled and unlabeled nodes. Self-training has emerged as a widely popular framework to leverage the abundance of unlabeled data, which expands the training set by assigning pseudo-labels to selected unlabeled nodes. Efforts have been made to develop various selection strategies based on confidence, information gain, etc. However, none of these methods takes into account the distribution shift between the training and testing node sets. The pseudo-labeling step may amplify this shift and even introduce new ones, hindering the effectiveness of self-training. Therefore, in this work, we explore the potential of explicitly bridging the distribution shift between the expanded training set and test set during self-training. To this end, we propose a novel Distribution-Consistent Graph Self-Training (DC-GST) framework to identify pseudo-labeled nodes that are both informative and capable of redeeming the distribution discrepancy and formulate it as a differentiable optimization task. A distribution-shift-aware edge predictor is further adopted to augment the graph and increase the model's generalizability in assigning pseudo labels. We evaluate our proposed method on four publicly available benchmark datasets and extensive experiments demonstrate that our framework consistently outperforms state-of-the-art baselines.

arXiv commentAccepted by WSDM 2024

References in corpus (11)

• Adam: A Method for Stochastic Optimization
• The Concrete Distribution: A Continuous Relaxation of Discrete Random Variables
• DropEdge: Towards Deep Graph Convolutional Networks on Node Classification
• Rethinking Pre-training and Self-training
• Robust Unsupervised Domain Adaptation for Neural Networks via Moment Alignment
• Be Confident! Towards Trustworthy Graph Neural Networks via Confidence Calibration
• Handling Distribution Shifts on Graphs: An Invariance Perspective
• Out-Of-Distribution Generalization on Graphs: A Survey
• Shift-Robust GNNs: Overcoming the Limitations of Localized Graph Training Data
• Recent Advances in Reliable Deep Graph Learning: Inherent Noise, Distribution Shift, and Adversarial Attack
• Toward Robust Graph Semi-Supervised Learning against Extreme Data Scarcity

Cited by in corpus (2)

• Counterfactual Learning on Graphs: A Survey
• HC-GST: Heterophily-aware Distribution Consistency based Graph Self-training