Deeper Insights into Graph Convolutional Networks for Semi-Supervised Learning

Abstract

Many interesting problems in machine learning are being revisited with new deep learning tools. For graph-based semisupervised learning, a recent important development is graph convolutional networks (GCNs), which nicely integrate local vertex features and graph topology in the convolutional layers. Although the GCN model compares favorably with other state-of-the-art methods, its mechanisms are not clear and it still requires a considerable amount of labeled data for validation and model selection. In this paper, we develop deeper insights into the GCN model and address its fundamental limits. First, we show that the graph convolution of the GCN model is actually a special form of Laplacian smoothing, which is the key reason why GCNs work, but it also brings potential concerns of over-smoothing with many convolutional layers. Second, to overcome the limits of the GCN model with shallow architectures, we propose both co-training and self-training approaches to train GCNs. Our approaches significantly improve GCNs in learning with very few labels, and exempt them from requiring additional labels for validation. Extensive experiments on benchmarks have verified our theory and proposals.

AAAI-2018 Oral Presentation

Cited by in corpus (72)

• Simplifying Graph Convolutional Networks
• Spatial-Temporal Graph ODE Networks for Traffic Flow Forecasting
• Scaling Graph Neural Networks with Approximate PageRank
• A Gentle Introduction to Deep Learning for Graphs
• Monocular Depth Estimation Based On Deep Learning: An Overview
• Graph-Based Deep Learning for Medical Diagnosis and Analysis: Past, Present and Future
• Multi-view Graph Contrastive Representation Learning for Drug-Drug Interaction Prediction
• A Note on Over-Smoothing for Graph Neural Networks
• Heterogeneous Graph Representation Learning with Relation Awareness
• Towards Deeper Graph Neural Networks with Differentiable Group Normalization
• TransGCN:Coupling Transformation Assumptions with Graph Convolutional Networks for Link Prediction
• Modeling Global and Local Node Contexts for Text Generation from Knowledge Graphs
• Should Graph Convolution Trust Neighbors? A Simple Causal Inference Method
• When Does Self-Supervision Help Graph Convolutional Networks?
• DropMessage: Unifying Random Dropping for Graph Neural Networks
• AdaGNN: Graph Neural Networks with Adaptive Frequency Response Filter
• Grapy-ML: Graph Pyramid Mutual Learning for Cross-dataset Human Parsing
• Self-supervised Consensus Representation Learning for Attributed Graph
• Flow Completion Network: Inferring the Fluid Dynamics from Incomplete Flow Information using Graph Neural Networks
• Zero-shot Node Classification with Decomposed Graph Prototype Network
• Propagation Tree Is Not Deep: Adaptive Graph Contrastive Learning Approach for Rumor Detection
• Meta-GNN: On Few-shot Node Classification in Graph Meta-learning
• AdaGCN: Adaboosting Graph Convolutional Networks into Deep Models
• Bag of Tricks for Node Classification with Graph Neural Networks
• Molecular Property Prediction: A Multilevel Quantum Interactions Modeling Perspective
• Improving Graph Attention Networks with Large Margin-based Constraints
• Understanding Negative Sampling in Graph Representation Learning
• ROD: Reception-aware Online Distillation for Sparse Graphs
• GLMNet: Graph Learning-Matching Networks for Feature Matching
• Gophormer: Ego-Graph Transformer for Node Classification
• Beyond Graph Convolutional Network: An Interpretable Regularizer-centered Optimization Framework
• Learning to Identify High Betweenness Centrality Nodes from Scratch: A Novel Graph Neural Network Approach
• Edge Representation Learning with Hypergraphs
• Graph-Based Global Reasoning Networks
• DEMO-Net: Degree-specific Graph Neural Networks for Node and Graph Classification
• Learning to Drop: Robust Graph Neural Network via Topological Denoising
• Node Similarity Preserving Graph Convolutional Networks
• Say No to the Discrimination: Learning Fair Graph Neural Networks with Limited Sensitive Attribute Information
• Revisiting Graph based Collaborative Filtering: A Linear Residual Graph Convolutional Network Approach
• Multiresolution Graph Transformers and Wavelet Positional Encoding for Learning Hierarchical Structures
• Signed Graph Diffusion Network
• RGAT: A Deeper Look into Syntactic Dependency Information for Coreference Resolution
• Self-supervised graph neural networks for accurate prediction of Néel temperature
• Dual ResGCN for Balanced Scene GraphGeneration
• A Hierarchy of Graph Neural Networks Based on Learnable Local Features
• Node Masking: Making Graph Neural Networks Generalize and Scale Better
• Weisfeiler and Lehman Go Paths: Learning Topological Features via Path Complexes
• Every Node Counts: Self-Ensembling Graph Convolutional Networks for Semi-Supervised Learning
• Graph Convolutional Network-based Suspicious Communication Pair Estimation for Industrial Control Systems
• Graph Ensemble Learning over Multiple Dependency Trees for Aspect-level Sentiment Classification
• Causal Discovery from Incomplete Data using An Encoder and Reinforcement Learning
• Graph Neural Lasso for Dynamic Network Regression
• Graph Neural Distance Metric Learning with Graph-Bert
• Higher-Order Implicit Fairing Networks for 3D Human Pose Estimation
• Adaptive Gaussian Processes on Graphs via Spectral Graph Wavelets
• GraLSP: Graph Neural Networks with Local Structural Patterns
• On Self-Distilling Graph Neural Network
• Beyond Homophily with Graph Echo State Networks
• LEReg: Empower Graph Neural Networks with Local Energy Regularization
• Multi-Zone Unit for Recurrent Neural Networks
• Tensor Graph Convolutional Networks for Text Classification
• G5: A Universal GRAPH-BERT for Graph-to-Graph Transfer and Apocalypse Learning
• On Local Aggregation in Heterophilic Graphs
• Semantic Graph Convolutional Network for Implicit Discourse Relation Classification
• FeatureNorm: L2 Feature Normalization for Dynamic Graph Embedding
• Fully Linear Graph Convolutional Networks for Semi-Supervised Learning and Clustering
• Label-Aware Graph Convolutional Networks
• Spectral Transform Forms Scalable Transformer
• Network representation learning: A macro and micro view
• Multiplex Bipartite Network Embedding using Dual Hypergraph Convolutional Networks
• Effective Eigendecomposition based Graph Adaptation for Heterophilic Networks
• Mutual Teaching for Graph Convolutional Networks