Distilling Self-Knowledge From Contrastive Links to Classify Graph Nodes Without Passing Messages

Abstract

Nowadays, Graph Neural Networks (GNNs) following the Message Passing paradigm become the dominant way to learn on graphic data. Models in this paradigm have to spend extra space to look up adjacent nodes with adjacency matrices and extra time to aggregate multiple messages from adjacent nodes. To address this issue, we develop a method called LinkDist that distils self-knowledge from connected node pairs into a Multi-Layer Perceptron (MLP) without the need to aggregate messages. Experiment with 8 real-world datasets shows the MLP derived from LinkDist can predict the label of a node without knowing its adjacencies but achieve comparable accuracy against GNNs in the contexts of semi- and full-supervised node classification. Moreover, LinkDist benefits from its Non-Message Passing paradigm that we can also distil self-knowledge from arbitrarily sampled node pairs in a contrastive way to further boost the performance of LinkDist.

9 pages, 2 figures, 4 tables

References in corpus (6)

• Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift
• Improving neural networks by preventing co-adaptation of feature detectors
• Inductive Representation Learning on Large Graphs
• Knowledge Distillation: A Survey
• Deep Graph Library: A Graph-Centric, Highly-Performant Package for Graph Neural Networks
• Layer Normalization

Cited by in corpus (2)

• Clarify Confused Nodes via Separated Learning
• Tackling the Local Bias in Federated Graph Learning