Simple and Deep Graph Convolutional Networks

Abstract

Graph convolutional networks (GCNs) are a powerful deep learning approach for graph-structured data. Recently, GCNs and subsequent variants have shown superior performance in various application areas on real-world datasets. Despite their success, most of the current GCN models are shallow, due to the {\em over-smoothing} problem. In this paper, we study the problem of designing and analyzing deep graph convolutional networks. We propose the GCNII, an extension of the vanilla GCN model with two simple yet effective techniques: {\em Initial residual} and {\em Identity mapping}. We provide theoretical and empirical evidence that the two techniques effectively relieves the problem of over-smoothing. Our experiments show that the deep GCNII model outperforms the state-of-the-art methods on various semi- and full-supervised tasks. Code is available at https://github.com/chennnM/GCNII .

ICML 2020

Cited by in corpus (88)

• Remote Sensing Cross-Modal Text-Image Retrieval Based on Global and Local Information
• Automated Self-Supervised Learning for Recommendation
• Combining Label Propagation and Simple Models Out-performs Graph Neural Networks
• Graph Attention Multi-Layer Perceptron
• Improving Fairness in Graph Neural Networks via Mitigating Sensitive Attribute Leakage
• On the Bottleneck of Graph Neural Networks and its Practical Implications
• A Comparison of Neural Network Architectures for Data-Driven Reduced-Order Modeling
• Unnoticeable Backdoor Attacks on Graph Neural Networks
• Should Graph Convolution Trust Neighbors? A Simple Causal Inference Method
• Regular Splitting Graph Network for 3D Human Pose Estimation
• Is Homophily a Necessity for Graph Neural Networks?
• AdaGNN: Graph Neural Networks with Adaptive Frequency Response Filter
• SE-GSL: A General and Effective Graph Structure Learning Framework through Structural Entropy Optimization
• Beyond Low-pass Filtering: Graph Convolutional Networks with Automatic Filtering
• Learning Strong Graph Neural Networks with Weak Information
• Is Heterophily A Real Nightmare For Graph Neural Networks To Do Node Classification?
• Model Degradation Hinders Deep Graph Neural Networks
• Company-as-Tribe: Company Financial Risk Assessment on Tribe-Style Graph with Hierarchical Graph Neural Networks
• UltraGCN: Ultra Simplification of Graph Convolutional Networks for Recommendation
• Designing the Topology of Graph Neural Networks: A Novel Feature Fusion Perspective
• CatGCN: Graph Convolutional Networks with Categorical Node Features
• FreeKD: Free-direction Knowledge Distillation for Graph Neural Networks
• Feature Overcorrelation in Deep Graph Neural Networks: A New Perspective
• Search to aggregate neighborhood for graph neural network
• DuETA: Traffic Congestion Propagation Pattern Modeling via Efficient Graph Learning for ETA Prediction at Baidu Maps
• New Benchmarks for Learning on Non-Homophilous Graphs
• Two Sides of the Same Coin: Heterophily and Oversmoothing in Graph Convolutional Neural Networks
• Meta-Weight Graph Neural Network: Push the Limits Beyond Global Homophily
• GCNH: A Simple Method For Representation Learning On Heterophilous Graphs
• Structure-Aware DropEdge Towards Deep Graph Convolutional Networks
• Local Augmentation for Graph Neural Networks
• Search to Capture Long-range Dependency with Stacking GNNs for Graph Classification
• PDE-GCN: Novel Architectures for Graph Neural Networks Motivated by Partial Differential Equations
• Spatial Heterophily Aware Graph Neural Networks
• On Provable Benefits of Depth in Training Graph Convolutional Networks
• Improving Graph Neural Networks with Simple Architecture Design
• Evaluating Deep Graph Neural Networks
• wsGAT: Weighted and Signed Graph Attention Networks for Link Prediction
• Residual Network and Embedding Usage: New Tricks of Node Classification with Graph Convolutional Networks
• Dynamic Relevance Learning for Few-Shot Object Detection
• Bridged-GNN: Knowledge Bridge Learning for Effective Knowledge Transfer
• Graph Neural Networks Inspired by Classical Iterative Algorithms
• p2pGNN: A Decentralized Graph Neural Network for Node Classification in Peer-to-Peer Networks
• Finding Heterophilic Neighbors via Confidence-based Subgraph Matching for Semi-supervised Node Classification
• How Framelets Enhance Graph Neural Networks
• FairSample: Training Fair and Accurate Graph Convolutional Neural Networks Efficiently
• Towards Deepening Graph Neural Networks: A GNTK-based Optimization Perspective
• Deepened Graph Auto-Encoders Help Stabilize and Enhance Link Prediction
• Clarify Confused Nodes via Separated Learning
• Optimization and Generalization Analysis of Transduction through Gradient Boosting and Application to Multi-scale Graph Neural Networks
• CogDL: A Comprehensive Library for Graph Deep Learning
• Label-Wise Graph Convolutional Network for Heterophilic Graphs
• MMGCN: Multimodal Fusion via Deep Graph Convolution Network for Emotion Recognition in Conversation
• Hyperbolic Deep Learning for Foundation Models: A Survey
• Tree Decomposed Graph Neural Network
• Co-attention Graph Pooling for Efficient Pairwise Graph Interaction Learning
• Isometric Transformation Invariant and Equivariant Graph Convolutional Networks
• Search For Deep Graph Neural Networks
• Implicit SVD for Graph Representation Learning
• Topological Regularization for Graph Neural Networks Augmentation
• KMF: Knowledge-Aware Multi-Faceted Representation Learning for Zero-Shot Node Classification
• When Contrastive Learning Meets Active Learning: A Novel Graph Active Learning Paradigm with Self-Supervision
• Quantized Convolutional Neural Networks Through the Lens of Partial Differential Equations
• SoGCN: Second-Order Graph Convolutional Networks
• EWasteNet: A Two-Stream Data Efficient Image Transformer Approach for E-Waste Classification
• Multi-task Self-distillation for Graph-based Semi-Supervised Learning
• Graph Neural Networks with Feature and Structure Aware Random Walk
• A Deep Graph Wavelet Convolutional Neural Network for Semi-supervised Node Classification
• Graph Traversal with Tensor Functionals: A Meta-Algorithm for Scalable Learning
• Stacked Graph Filter
• Asymmetric Graph Representation Learning
• AKE-GNN: Effective Graph Learning with Adaptive Knowledge Exchange
• Multi-Level Attention Pooling for Graph Neural Networks: Unifying Graph Representations with Multiple Localities
• Self-Supervised Deep Graph Embedding with High-Order Information Fusion for Community Discovery
• Simplification of Graph Convolutional Networks: A Matrix Factorization-based Perspective
• Prior Signal Editing for Graph Filter Posterior Fairness Constraints
• FDGATII : Fast Dynamic Graph Attention with Initial Residual and Identity Mapping
• Scaling Up Graph Neural Networks Via Graph Coarsening
• Towards Self-Explainable Graph Neural Network
• Fast Graph Learning with Unique Optimal Solutions
• Statistical Estimation from Dependent Data
• Rotation Equivariant 3D Hand Mesh Generation from a Single RGB Image
• Automated Graph Learning via Population Based Self-Tuning GCN
• New Insights into Graph Convolutional Networks using Neural Tangent Kernels
• Multi-hop Graph Convolutional Network with High-order Chebyshev Approximation for Text Reasoning
• SAS: A Simple, Accurate and Scalable Node Classification Algorithm
• Residual Enhanced Multi-Hypergraph Neural Network
• Cross-lingual Transfer for Text Classification with Dictionary-based Heterogeneous Graph