LTE4G: Long-Tail Experts for Graph Neural Networks

Abstract

Existing Graph Neural Networks (GNNs) usually assume a balanced situation where both the class distribution and the node degree distribution are balanced. However, in real-world situations, we often encounter cases where a few classes (i.e., head class) dominate other classes (i.e., tail class) as well as in the node degree perspective, and thus naively applying existing GNNs eventually fall short of generalizing to the tail cases. Although recent studies proposed methods to handle long-tail situations on graphs, they only focus on either the class long-tailedness or the degree long-tailedness. In this paper, we propose a novel framework for training GNNs, called Long-Tail Experts for Graphs (LTE4G), which jointly considers the class long-tailedness, and the degree long-tailedness for node classification. The core idea is to assign an expert GNN model to each subset of nodes that are split in a balanced manner considering both the class and degree long-tailedness. After having trained an expert for each balanced subset, we adopt knowledge distillation to obtain two class-wise students, i.e., Head class student and Tail class student, each of which is responsible for classifying nodes in the head classes and tail classes, respectively. We demonstrate that LTE4G outperforms a wide range of state-of-the-art methods in node classification evaluated on both manual and natural imbalanced graphs. The source code of LTE4G can be found at https://github.com/SukwonYun/LTE4G.

Accepted by CIKM 2022, the code is available at https://github.com/SukwonYun/LTE4G

References in corpus (5)

• Distilling the Knowledge in a Neural Network
• GraphSMOTE: Imbalanced Node Classification on Graphs with Graph Neural Networks
• Investigating and Mitigating Degree-Related Biases in Graph Convolutional Networks
• Graph-based Anomaly Detection and Description: A Survey
• ImGAGN:Imbalanced Network Embedding via Generative Adversarial Graph Networks