Two Sides of the Same Coin: Heterophily and Oversmoothing in Graph Convolutional Neural Networks

Abstract

In node classification tasks, graph convolutional neural networks (GCNs) have demonstrated competitive performance over traditional methods on diverse graph data. However, it is known that the performance of GCNs degrades with increasing number of layers (oversmoothing problem) and recent studies have also shown that GCNs may perform worse in heterophilous graphs, where neighboring nodes tend to belong to different classes (heterophily problem). These two problems are usually viewed as unrelated, and thus are studied independently, often at the graph filter level from a spectral perspective. We are the first to take a unified perspective to jointly explain the oversmoothing and heterophily problems at the node level. Specifically, we profile the nodes via two quantitative metrics: the relative degree of a node (compared to its neighbors) and the node-level heterophily. Our theory shows that the interplay of these two profiling metrics defines three cases of node behaviors, which explain the oversmoothing and heterophily problems jointly and can predict the performance of GCNs. Based on insights from our theory, we show theoretically and empirically the effectiveness of two strategies: structure-based edge correction, which learns corrected edge weights from structural properties (i.e., degrees), and feature-based edge correction, which learns signed edge weights from node features. Compared to other approaches, which tend to handle well either heterophily or oversmoothing, we show that {our model, GGCN}, which incorporates the two strategies performs well in both problems.

arXiv commentAccepted to ICDM 2022, including 14-page supplement

References in corpus (18)

• Semi-Supervised Classification with Graph Convolutional Networks
• Assortative mixing in networks
• Inductive Representation Learning on Large Graphs
• Neural Message Passing for Quantum Chemistry
• Simplifying Graph Convolutional Networks
• Representation Learning on Graphs with Jumping Knowledge Networks
• DropEdge: Towards Deep Graph Convolutional Networks on Node Classification
• Pitfalls of Graph Neural Network Evaluation
• Simple and Deep Graph Convolutional Networks
• PairNorm: Tackling Oversmoothing in GNNs
• Geom-GCN: Geometric Graph Convolutional Networks
• Large Scale Learning on Non-Homophilous Graphs: New Benchmarks and Strong Simple Methods
• Non-Local Graph Neural Networks
• Improving Graph Attention Networks with Large Margin-based Constraints
• Understanding and Resolving Performance Degradation in Graph Convolutional Networks
• Beyond Low-frequency Information in Graph Convolutional Networks
• Graph Convolution for Semi-Supervised Classification: Improved Linear Separability and Out-of-Distribution Generalization
• Graph Neural Networks with Heterophily

Cited by in corpus (9)

• Is Heterophily A Real Nightmare For Graph Neural Networks To Do Node Classification?
• New Benchmarks for Learning on Non-Homophilous Graphs
• On Provable Benefits of Depth in Training Graph Convolutional Networks
• Evaluating Deep Graph Neural Networks
• Graph Convolution for Semi-Supervised Classification: Improved Linear Separability and Out-of-Distribution Generalization
• Distance-wise Prototypical Graph Neural Network in Node Imbalance Classification
• DyFormer: A Scalable Dynamic Graph Transformer with Provable Benefits on Generalization Ability
• Tree Decomposed Graph Neural Network
• An Empirical Study of Graph Contrastive Learning