Bag of Tricks for Node Classification with Graph Neural Networks

Abstract

Over the past few years, graph neural networks (GNN) and label propagation-based methods have made significant progress in addressing node classification tasks on graphs. However, in addition to their reliance on elaborate architectures and algorithms, there are several key technical details that are frequently overlooked, and yet nonetheless can play a vital role in achieving satisfactory performance. In this paper, we first summarize a series of existing tricks-of-the-trade, and then propose several new ones related to label usage, loss function formulation, and model design that can significantly improve various GNN architectures. We empirically evaluate their impact on final node classification accuracy by conducting ablation studies and demonstrate consistently-improved performance, often to an extent that outweighs the gains from more dramatic changes in the underlying GNN architecture. Notably, many of the top-ranked models on the Open Graph Benchmark (OGB) leaderboard and KDDCUP 2021 Large-Scale Challenge MAG240M-LSC benefit from these techniques we initiated.

References in corpus (23)

• Semi-Supervised Classification with Graph Convolutional Networks
• Inductive Representation Learning on Large Graphs
• Neural Message Passing for Quantum Chemistry
• Generalized Cross Entropy Loss for Training Deep Neural Networks with Noisy Labels
• Deep Graph Library: A Graph-Centric, Highly-Performant Package for Graph Neural Networks
• DropEdge: Towards Deep Graph Convolutional Networks on Node Classification
• Open Graph Benchmark: Datasets for Machine Learning on Graphs
• FastGCN: Fast Learning with Graph Convolutional Networks via Importance Sampling
• Deeper Insights into Graph Convolutional Networks for Semi-Supervised Learning
• How Powerful are Graph Neural Networks?
• DeeperGCN: All You Need to Train Deeper GCNs
• SIGN: Scalable Inception Graph Neural Networks
• Beyond Homophily in Graph Neural Networks: Current Limitations and Effective Designs
• Relational Graph Attention Networks
• OGB-LSC: A Large-Scale Challenge for Machine Learning on Graphs
• Geom-GCN: Geometric Graph Convolutional Networks
• Combining Label Propagation and Simple Models Out-performs Graph Neural Networks
• Masked Label Prediction: Unified Message Passing Model for Semi-Supervised Classification
• Unifying Graph Convolutional Neural Networks and Label Propagation
• Layer-Dependent Importance Sampling for Training Deep and Large Graph Convolutional Networks
• Understanding Negative Sampling in Graph Representation Learning
• Graph Neural Networks Inspired by Classical Iterative Algorithms
• Why Propagate Alone? Parallel Use of Labels and Features on Graphs

Cited by in corpus (7)

• Graph Attention Multi-Layer Perceptron
• Trustworthy Graph Neural Networks: Aspects, Methods and Trends
• Label Information Enhanced Fraud Detection against Low Homophily in Graphs
• How Attentive are Graph Attention Networks?
• Residual Network and Embedding Usage: New Tricks of Node Classification with Graph Convolutional Networks
• Large-scale graph representation learning with very deep GNNs and self-supervision
• Does your graph need a confidence boost? Convergent boosted smoothing on graphs with tabular node features