Revisiting Graph Convolutional Network on Semi-Supervised Node Classification from an Optimization Perspective

Abstract

Graph convolutional networks (GCNs) have achieved promising performance on various graph-based tasks. However they suffer from over-smoothing when stacking more layers. In this paper, we present a quantitative study on this observation and develop novel insights towards the deeper GCN. First, we interpret the current graph convolutional operations from an optimization perspective and argue that over-smoothing is mainly caused by the naive first-order approximation of the solution to the optimization problem. Subsequently, we introduce two metrics to measure the over-smoothing on node-level tasks. Specifically, we calculate the fraction of the pairwise distance between connected and disconnected nodes to the overall distance respectively. Based on our theoretical and empirical analysis, we establish a universal theoretical framework of GCN from an optimization perspective and derive a novel convolutional kernel named GCN+ which has lower parameter amount while relieving the over-smoothing inherently. Extensive experiments on real-world datasets demonstrate the superior performance of GCN+ over state-of-the-art baseline methods on the node classification tasks.

References in corpus (20)

• Adam: A Method for Stochastic Optimization
• PyTorch: An Imperative Style, High-Performance Deep Learning Library
• Inductive Representation Learning on Large Graphs
• Fast Graph Representation Learning with PyTorch Geometric
• Simplifying Graph Convolutional Networks
• Representation Learning on Graphs with Jumping Knowledge Networks
• Variational Graph Auto-Encoders
• DropEdge: Towards Deep Graph Convolutional Networks on Node Classification
• DeepInf: Social Influence Prediction with Deep Learning
• Predicting multicellular function through multi-layer tissue networks
• Towards Deeper Graph Neural Networks
• Open Graph Benchmark: Datasets for Machine Learning on Graphs
• Gated Graph Sequence Neural Networks
• DeeperGCN: All You Need to Train Deeper GCNs
• PairNorm: Tackling Oversmoothing in GNNs
• DeepGCNs: Can GCNs Go as Deep as CNNs?
• A Note on Over-Smoothing for Graph Neural Networks
• Revisiting Over-smoothing in Deep GCNs
• Continuous Graph Neural Networks
• Variational Graph Recurrent Neural Networks

Cited by in corpus (4)

• Graph Neural Networks Inspired by Classical Iterative Algorithms
• Cold Brew: Distilling Graph Node Representations with Incomplete or Missing Neighborhoods
• Implicit vs Unfolded Graph Neural Networks
• Does your graph need a confidence boost? Convergent boosted smoothing on graphs with tabular node features