Masked Label Prediction: Unified Message Passing Model for Semi-Supervised Classification

Abstract

Graph neural network (GNN) and label propagation algorithm (LPA) are both message passing algorithms, which have achieved superior performance in semi-supervised classification. GNN performs feature propagation by a neural network to make predictions, while LPA uses label propagation across graph adjacency matrix to get results. However, there is still no effective way to directly combine these two kinds of algorithms. To address this issue, we propose a novel Unified Message Passaging Model (UniMP) that can incorporate feature and label propagation at both training and inference time. First, UniMP adopts a Graph Transformer network, taking feature embedding and label embedding as input information for propagation. Second, to train the network without overfitting in self-loop input label information, UniMP introduces a masked label prediction strategy, in which some percentage of input label information are masked at random, and then predicted. UniMP conceptually unifies feature propagation and label propagation and is empirically powerful. It obtains new state-of-the-art semi-supervised classification results in Open Graph Benchmark (OGB).

arXiv comment7 pages, 3 figures and 8 tables; Accepted by IJCAI 2021

References in corpus (17)

• Semi-Supervised Classification with Graph Convolutional Networks
• Inductive Representation Learning on Large Graphs
• Cluster-GCN: An Efficient Algorithm for Training Deep and Large Graph Convolutional Networks
• Representation Learning on Graphs with Jumping Knowledge Networks
• Learning Convolutional Neural Networks for Graphs
• Learning to Propagate Labels: Transductive Propagation Network for Few-shot Learning
• Open Graph Benchmark: Datasets for Machine Learning on Graphs
• How Powerful are Graph Neural Networks?
• GraphSAINT: Graph Sampling Based Inductive Learning Method
• DeeperGCN: All You Need to Train Deeper GCNs
• GaAN: Gated Attention Networks for Learning on Large and Spatiotemporal Graphs
• Benchmarking Graph Neural Networks
• NetGAN: Generating Graphs via Random Walks
• LanczosNet: Multi-Scale Deep Graph Convolutional Networks
• Unifying Graph Convolutional Neural Networks and Label Propagation
• Joint Embedding of Words and Labels for Text Classification
• Bandit Samplers for Training Graph Neural Networks

Cited by in corpus (20)

• Do Transformers Really Perform Bad for Graph Representation?
• OGB-LSC: A Large-Scale Challenge for Machine Learning on Graphs
• Combining Label Propagation and Simple Models Out-performs Graph Neural Networks
• Graph Attention Multi-Layer Perceptron
• How Attentive are Graph Attention Networks?
• Flow Network based Generative Models for Non-Iterative Diverse Candidate Generation
• Bag of Tricks for Node Classification with Graph Neural Networks
• Scalable and Adaptive Graph Neural Networks with Self-Label-Enhanced training
• Privacy-Preserving Individual-Level COVID-19 Infection Prediction via Federated Graph Learning
• Residual Network and Embedding Usage: New Tricks of Node Classification with Graph Convolutional Networks
• Deep Graph Neural Networks with Shallow Subgraph Samplers
• Neural Link Prediction with Walk Pooling
• Extract the Knowledge of Graph Neural Networks and Go Beyond it: An Effective Knowledge Distillation Framework
• Why Propagate Alone? Parallel Use of Labels and Features on Graphs
• Lipschitz Normalization for Self-Attention Layers with Application to Graph Neural Networks
• Multi-Grid Graph Neural Networks with Self-Attention for Computational Mechanics
• Graph Belief Propagation Networks
• On Local Aggregation in Heterophilic Graphs
• Sequential Aggregation and Rematerialization: Distributed Full-batch Training of Graph Neural Networks on Large Graphs
• Graph Neural Network Training with Data Tiering