Two Sides of the Same Coin: Heterophily and Oversmoothing in Graph Convolutional Neural Networks

Abstract

In node classification tasks, graph convolutional neural networks (GCNs) have demonstrated competitive performance over traditional methods on diverse graph data. However, it is known that the performance of GCNs degrades with increasing number of layers (oversmoothing problem) and recent studies have also shown that GCNs may perform worse in heterophilous graphs, where neighboring nodes tend to belong to different classes (heterophily problem). These two problems are usually viewed as unrelated, and thus are studied independently, often at the graph filter level from a spectral perspective. We are the first to take a unified perspective to jointly explain the oversmoothing and heterophily problems at the node level. Specifically, we profile the nodes via two quantitative metrics: the relative degree of a node (compared to its neighbors) and the node-level heterophily. Our theory shows that the interplay of these two profiling metrics defines three cases of node behaviors, which explain the oversmoothing and heterophily problems jointly and can predict the performance of GCNs. Based on insights from our theory, we show theoretically and empirically the effectiveness of two strategies: structure-based edge correction, which learns corrected edge weights from structural properties (i.e., degrees), and feature-based edge correction, which learns signed edge weights from node features. Compared to other approaches, which tend to handle well either heterophily or oversmoothing, we show that {our model, GGCN}, which incorporates the two strategies performs well in both problems.

Accepted to ICDM 2022, including 14-page supplement

References in corpus (8)

• Semi-Supervised Classification with Graph Convolutional Networks
• Simplifying Graph Convolutional Networks
• Simple and Deep Graph Convolutional Networks
• Geom-GCN: Geometric Graph Convolutional Networks
• Large Scale Learning on Non-Homophilous Graphs: New Benchmarks and Strong Simple Methods
• Improving Graph Attention Networks with Large Margin-based Constraints
• Beyond Low-frequency Information in Graph Convolutional Networks
• Graph Convolution for Semi-Supervised Classification: Improved Linear Separability and Out-of-Distribution Generalization

Cited by in corpus (9)

• Is Heterophily A Real Nightmare For Graph Neural Networks To Do Node Classification?
• New Benchmarks for Learning on Non-Homophilous Graphs
• On Provable Benefits of Depth in Training Graph Convolutional Networks
• Evaluating Deep Graph Neural Networks
• Graph Convolution for Semi-Supervised Classification: Improved Linear Separability and Out-of-Distribution Generalization
• Distance-wise Prototypical Graph Neural Network in Node Imbalance Classification
• DyFormer: A Scalable Dynamic Graph Transformer with Provable Benefits on Generalization Ability
• Tree Decomposed Graph Neural Network
• An Empirical Study of Graph Contrastive Learning