Structure-Aware Label Smoothing for Graph Neural Networks

Abstract

Representing a label distribution as a one-hot vector is a common practice in training node classification models. However, the one-hot representation may not adequately reflect the semantic characteristics of a node in different classes, as some nodes may be semantically close to their neighbors in other classes. It would cause over-confidence since the models are encouraged to assign full probabilities when classifying every node. While training models with label smoothing can ease this problem to some degree, it still fails to capture the nodes' semantic characteristics implied by the graph structures. In this work, we propose a novel SALS (\textit{Structure-Aware Label Smoothing}) method as an enhancement component to popular node classification models. SALS leverages the graph structures to capture the semantic correlations between the connected nodes and generate the structure-aware label distribution to replace the original one-hot label vectors, thus improving the node classification performance without inference costs. Extensive experiments on seven node classification benchmark datasets reveal the effectiveness of our SALS on improving both transductive and inductive node classification. Empirical results show that SALS is superior to the label smoothing method and enhances the node classification models to outperform the baseline methods.

References in corpus (18)

• Semi-Supervised Classification with Graph Convolutional Networks
• Inductive Representation Learning on Large Graphs
• Spectral Networks and Locally Connected Networks on Graphs
• On Calibration of Modern Neural Networks
• Molecular Graph Convolutions: Moving Beyond Fingerprints
• Graph Neural Networks: A Review of Methods and Applications
• Cluster-GCN: An Efficient Algorithm for Training Deep and Large Graph Convolutional Networks
• Representation Learning on Graphs with Jumping Knowledge Networks
• When Does Label Smoothing Help?
• DropEdge: Towards Deep Graph Convolutional Networks on Node Classification
• Pitfalls of Graph Neural Network Evaluation
• GraphSAINT: Graph Sampling Based Inductive Learning Method
• GaAN: Gated Attention Networks for Learning on Large and Spatiotemporal Graphs
• SIGN: Scalable Inception Graph Neural Networks
• GMNN: Graph Markov Neural Networks
• DeepGCNs: Can GCNs Go as Deep as CNNs?
• Unifying Graph Convolutional Neural Networks and Label Propagation
• Graph Star Net for Generalized Multi-Task Learning