Shift-Robust GNNs: Overcoming the Limitations of Localized Graph Training Data

Abstract

There has been a recent surge of interest in designing Graph Neural Networks (GNNs) for semi-supervised learning tasks. Unfortunately this work has assumed that the nodes labeled for use in training were selected uniformly at random (i.e. are an IID sample). However in many real world scenarios gathering labels for graph nodes is both expensive and inherently biased -- so this assumption can not be met. GNNs can suffer poor generalization when this occurs, by overfitting to superfluous regularities present in the training data. In this work we present a method, Shift-Robust GNN (SR-GNN), designed to account for distributional differences between biased training data and the graph's true inference distribution. SR-GNN adapts GNN models for the presence of distributional shifts between the nodes which have had labels provided for training and the rest of the dataset. We illustrate the effectiveness of SR-GNN in a variety of experiments with biased training datasets on common GNN benchmark datasets for semi-supervised learning, where we see that SR-GNN outperforms other GNN baselines by accuracy, eliminating at least (~40%) of the negative effects introduced by biased training data. On the largest dataset we consider, ogb-arxiv, we observe an 2% absolute improvement over the baseline and reduce 30% of the negative effects.

arXiv commentNeurIPS 2021

References in corpus (16)

• Adam: A Method for Stochastic Optimization
• Inductive Representation Learning on Large Graphs
• Neural Message Passing for Quantum Chemistry
• Learning Transferable Features with Deep Adaptation Networks
• Simplifying Graph Convolutional Networks
• Domain Adaptation: Learning Bounds and Algorithms
• Open Graph Benchmark: Datasets for Machine Learning on Graphs
• Scaling Graph Neural Networks with Approximate PageRank
• An introduction to domain adaptation and transfer learning
• Central Moment Discrepancy (CMD) for Domain-Invariant Representation Learning
• Strategies for Pre-training Graph Neural Networks
• Network Together: Node Classification via Cross network Deep Network Embedding
• Robust Unsupervised Domain Adaptation for Neural Networks via Moment Alignment
• Machine Learning on Graphs: A Model and Comprehensive Taxonomy
• Graph Clustering with Graph Neural Networks
• Grale: Designing Networks for Graph Learning

Cited by in corpus (3)

• Distribution Consistency based Self-Training for Graph Neural Networks with Sparse Labels
• CDMPP: A Device-Model Agnostic Framework for Latency Prediction of Tensor Programs
• HC-GST: Heterophily-aware Distribution Consistency based Graph Self-training