DropEdge: Towards Deep Graph Convolutional Networks on Node Classification

Abstract

\emph{Over-fitting} and \emph{over-smoothing} are two main obstacles of developing deep Graph Convolutional Networks (GCNs) for node classification. In particular, over-fitting weakens the generalization ability on small dataset, while over-smoothing impedes model training by isolating output representations from the input features with the increase in network depth. This paper proposes DropEdge, a novel and flexible technique to alleviate both issues. At its core, DropEdge randomly removes a certain number of edges from the input graph at each training epoch, acting like a data augmenter and also a message passing reducer. Furthermore, we theoretically demonstrate that DropEdge either reduces the convergence speed of over-smoothing or relieves the information loss caused by it. More importantly, our DropEdge is a general skill that can be equipped with many other backbone models (e.g. GCN, ResGCN, GraphSAGE, and JKNet) for enhanced performance. Extensive experiments on several benchmarks verify that DropEdge consistently improves the performance on a variety of both shallow and deep GCNs. The effect of DropEdge on preventing over-smoothing is empirically visualized and validated as well. Codes are released on~\url{https://github.com/DropEdge/DropEdge}.

13 pages

References in corpus (2)

• Improving neural networks by preventing co-adaptation of feature detectors
• Deep Convolutional Networks on Graph-Structured Data

Cited by in corpus (145)

• Graph Self-Supervised Learning: A Survey
• Open Graph Benchmark: Datasets for Machine Learning on Graphs
• Self-Supervised Graph Transformer on Large-Scale Molecular Data
• Deep Graph Contrastive Representation Learning
• SIGN: Scalable Inception Graph Neural Networks
• Graph Neural Networks: Taxonomy, Advances and Trends
• Differentiable Graph Module (DGM) for Graph Convolutional Networks
• Combining Label Propagation and Simple Models Out-performs Graph Neural Networks
• A Note on Over-Smoothing for Graph Neural Networks
• Handling Missing Data with Graph Representation Learning
• On the Bottleneck of Graph Neural Networks and its Practical Implications
• Graph Rationalization with Environment-based Augmentations
• MGDCF: Distance Learning via Markov Graph Diffusion for Neural Collaborative Filtering
• Should Graph Convolution Trust Neighbors? A Simple Causal Inference Method
• DropMessage: Unifying Random Dropping for Graph Neural Networks
• Implicit Graph Neural Networks
• Bi-GCN: Binary Graph Convolutional Network
• Revisiting Over-smoothing in Deep GCNs
• AdaGNN: Graph Neural Networks with Adaptive Frequency Response Filter
• SE-GSL: A General and Effective Graph Structure Learning Framework through Structural Entropy Optimization
• Uncertainty Aware Semi-Supervised Learning on Graph Data
• Scalable Graph Neural Networks via Bidirectional Propagation
• CCGL: Contrastive Cascade Graph Learning
• Graph Convolutional Neural Networks with Node Transition Probability-based Message Passing and DropNode Regularization
• Model Degradation Hinders Deep Graph Neural Networks
• How Attentive are Graph Attention Networks?
• HMSG: Heterogeneous Graph Neural Network based on Metapath Subgraph Learning
• Dirichlet Energy Constrained Learning for Deep Graph Neural Networks
• Semi-Supervised Hierarchical Graph Classification
• Bridging the Gap between Spatial and Spectral Domains: A Survey on Graph Neural Networks
• Designing the Topology of Graph Neural Networks: A Novel Feature Fusion Perspective
• Tensor Graph Convolutional Networks for Multi-relational and Robust Learning
• Large-Scale Representation Learning on Graphs via Bootstrapping
• Feature Overcorrelation in Deep Graph Neural Networks: A New Perspective
• Multi-domain Recommendation with Embedding Disentangling and Domain Alignment
• Stochastic Graph Neural Networks
• Interpreting and Unifying Graph Neural Networks with An Optimization Framework
• Bag of Tricks for Node Classification with Graph Neural Networks
• Search to aggregate neighborhood for graph neural network
• Graph Jigsaw Learning for Cartoon Face Recognition
• Understanding and Resolving Performance Degradation in Graph Convolutional Networks
• Two Sides of the Same Coin: Heterophily and Oversmoothing in Graph Convolutional Neural Networks
• Graph Random Neural Network for Semi-Supervised Learning on Graphs
• Learning Sparse Interaction Graphs of Partially Detected Pedestrians for Trajectory Prediction
• GraphCrop: Subgraph Cropping for Graph Classification
• Structure-Aware DropEdge Towards Deep Graph Convolutional Networks
• Degree-Quant: Quantization-Aware Training for Graph Neural Networks
• Hyperbolic Geometric Graph Representation Learning for Hierarchy-imbalance Node Classification
• Local Augmentation for Graph Neural Networks
• Edge Representation Learning with Hypergraphs
• PDE-GCN: Novel Architectures for Graph Neural Networks Motivated by Partial Differential Equations
• Improving Graph Neural Networks with Simple Architecture Design
• On Provable Benefits of Depth in Training Graph Convolutional Networks
• Dissecting the Diffusion Process in Linear Graph Convolutional Networks
• Neural message passing for joint paratope-epitope prediction
• TeMP: Temporal Message Passing for Temporal Knowledge Graph Completion
• Evaluating Deep Graph Neural Networks
• Learning to Drop: Robust Graph Neural Network via Topological Denoising
• Node Similarity Preserving Graph Convolutional Networks
• SSFG: Stochastically Scaling Features and Gradients for Regularizing Graph Convolutional Networks
• Sampling methods for efficient training of graph convolutional networks: A survey
• Deep Graph Neural Networks with Shallow Subgraph Samplers
• NODE-SELECT: A Graph Neural Network Based On A Selective Propagation Technique
• JuryGCN: Quantifying Jackknife Uncertainty on Graph Convolutional Networks
• Graph Neural Networks Inspired by Classical Iterative Algorithms
• TestRank: Bringing Order into Unlabeled Test Instances for Deep Learning Tasks
• Equivariant Subgraph Aggregation Networks
• Ripple Walk Training: A Subgraph-based training framework for Large and Deep Graph Neural Network
• Training Robust Graph Neural Networks with Topology Adaptive Edge Dropping
• DGDNN: Decoupled Graph Diffusion Neural Network for Stock Movement Prediction
• Distribution Consistency based Self-Training for Graph Neural Networks with Sparse Labels
• Robust Node Classification on Graphs: Jointly from Bayesian Label Transition and Topology-based Label Propagation
• Towards Open-World Feature Extrapolation: An Inductive Graph Learning Approach
• Revisiting Graph Convolutional Network on Semi-Supervised Node Classification from an Optimization Perspective
• Finding Heterophilic Neighbors via Confidence-based Subgraph Matching for Semi-supervised Node Classification
• Graph Posterior Network: Bayesian Predictive Uncertainty for Node Classification
• You are AllSet: A Multiset Function Framework for Hypergraph Neural Networks
• A Robust Hierarchical Graph Convolutional Network Model for Collaborative Filtering
• Towards Deepening Graph Neural Networks: A GNTK-based Optimization Perspective
• Deconvolutional Networks on Graph Data
• Node Masking: Making Graph Neural Networks Generalize and Scale Better
• Rethinking Graph Neural Architecture Search from Message-passing
• Weisfeiler and Lehman Go Paths: Learning Topological Features via Path Complexes
• Item Graph Convolution Collaborative Filtering for Inductive Recommendations
• Deoscillated Graph Collaborative Filtering
• Attentive Graph Neural Networks for Few-Shot Learning
• Deepened Graph Auto-Encoders Help Stabilize and Enhance Link Prediction
• GRADE: Automatic Graph-Enhanced Coherence Metric for Evaluating Open-Domain Dialogue Systems
• Semi-supervised Anomaly Detection on Attributed Graphs
• Accelerating SpMM Kernel with Cache-First Edge Sampling for Graph Neural Networks
• A pipeline for fair comparison of graph neural networks in node classification tasks
• R-Mixup: Riemannian Mixup for Biological Networks
• CogDL: A Comprehensive Library for Graph Deep Learning
• Optimization and Generalization Analysis of Transduction through Gradient Boosting and Application to Multi-scale Graph Neural Networks
• Accelerating Backward Aggregation in GCN Training with Execution Path Preparing on GPUs
• Reconstruction for Powerful Graph Representations
• Capsule Graph Neural Networks with EM Routing
• On Self-Distilling Graph Neural Network
• Search For Deep Graph Neural Networks
• Breaking the Expressive Bottlenecks of Graph Neural Networks
• HighwayGraph: Modelling Long-distance Node Relations for Improving General Graph Neural Network
• TransCamP: Graph Transformer for 6-DoF Camera Pose Estimation
• DeepHGCN: Toward Deeper Hyperbolic Graph Convolutional Networks
• TANGNN: a Concise, Scalable and Effective Graph Neural Networks with Top-m Attention Mechanism for Graph Representation Learning
• Multi-representations Space Separation based Graph-level Anomaly-aware Detection
• A Hard Label Black-box Adversarial Attack Against Graph Neural Networks
• High-Order Relation Construction and Mining for Graph Matching
• LEReg: Empower Graph Neural Networks with Local Energy Regularization
• Data Augmentation for Graph Convolutional Network on Semi-Supervised Classification
• Neuron with Steady Response Leads to Better Generalization
• Generalized Embedding Machines for Recommender Systems
• Structure-Aware Label Smoothing for Graph Neural Networks
• Multi-task Self-distillation for Graph-based Semi-Supervised Learning
• Cyclic Label Propagation for Graph Semi-supervised Learning
• SoGCN: Second-Order Graph Convolutional Networks
• A Deep Graph Wavelet Convolutional Neural Network for Semi-supervised Node Classification
• Asymmetric Graph Representation Learning
• Multi-Level Attention Pooling for Graph Neural Networks: Unifying Graph Representations with Multiple Localities
• G-SemTMO: Tone Mapping with a Trainable Semantic Graph
• BiTe-GCN: A New GCN Architecture via BidirectionalConvolution of Topology and Features on Text-Rich Networks
• Improving compute efficacy frontiers with SliceOut
• Self-Supervised Deep Graph Embedding with High-Order Information Fusion for Community Discovery
• Eigen-GNN: A Graph Structure Preserving Plug-in for GNNs
• Improving the Long-Range Performance of Gated Graph Neural Networks
• AKE-GNN: Effective Graph Learning with Adaptive Knowledge Exchange
• Dual GNNs: Graph Neural Network Learning with Limited Supervision
• Which Samples Should be Learned First: Easy or Hard?
• Curvature Graph Neural Network
• Automated Graph Learning via Population Based Self-Tuning GCN
• Training Graph Neural Networks by Graphon Estimation
• On the Global Self-attention Mechanism for Graph Convolutional Networks
• DeGNN: Characterizing and Improving Graph Neural Networks with Graph Decomposition
• Heuristic Semi-Supervised Learning for Graph Generation Inspired by Electoral College
• Exploring Graph Classification Techniques Under Low Data Constraints: A Comprehensive Study
• Hierarchical graph sampling based minibatch learning with chain preservation and variance reduction
• Contributions to Representation Learning with Graph Autoencoders and Applications to Music Recommendation
• FeatureNorm: L2 Feature Normalization for Dynamic Graph Embedding
• LSP : Acceleration and Regularization of Graph Neural Networks via Locality Sensitive Pruning of Graphs
• Implicit vs Unfolded Graph Neural Networks
• Multi-hop Graph Convolutional Network with High-order Chebyshev Approximation for Text Reasoning
• Scaling Up Graph Neural Networks Via Graph Coarsening
• Non-Recursive Graph Convolutional Networks
• Interferometric Graph Transform for Community Labeling
• HittER: Hierarchical Transformers for Knowledge Graph Embeddings
• Learnt Sparsification for Interpretable Graph Neural Networks