DeepGCNs: Can GCNs Go as Deep as CNNs?

Abstract

Convolutional Neural Networks (CNNs) achieve impressive performance in a wide variety of fields. Their success benefited from a massive boost when very deep CNN models were able to be reliably trained. Despite their merits, CNNs fail to properly address problems with non-Euclidean data. To overcome this challenge, Graph Convolutional Networks (GCNs) build graphs to represent non-Euclidean data, borrow concepts from CNNs, and apply them in training. GCNs show promising results, but they are usually limited to very shallow models due to the vanishing gradient problem. As a result, most state-of-the-art GCN models are no deeper than 3 or 4 layers. In this work, we present new ways to successfully train very deep GCNs. We do this by borrowing concepts from CNNs, specifically residual/dense connections and dilated convolutions, and adapting them to GCN architectures. Extensive experiments show the positive effect of these deep GCN frameworks. Finally, we use these new concepts to build a very deep 56-layer GCN, and show how it significantly boosts performance (+3.7% mIoU over state-of-the-art) in the task of point cloud semantic segmentation. We believe that the community can greatly benefit from this work, as it opens up many opportunities for advancing GCN-based research.

arXiv commentFirst two authors contributed equally. Accepted to ICCV'19 as oral presentation

References in corpus (11)

• Semi-Supervised Classification with Graph Convolutional Networks
• Rethinking Atrous Convolution for Semantic Image Segmentation
• Multi-Scale Context Aggregation by Dilated Convolutions
• Graph Neural Networks: A Review of Methods and Applications
• Representation Learning on Graphs with Jumping Knowledge Networks
• Joint 2D-3D-Semantic Data for Indoor Scene Understanding
• Predicting multicellular function through multi-layer tissue networks
• ScanNet: Richly-annotated 3D Reconstructions of Indoor Scenes
• Gated Graph Sequence Neural Networks
• How Powerful are Graph Neural Networks?
• Dynamic Graph CNN for Learning on Point Clouds

Cited by in corpus (48)

• Graph Neural Networks: A Review of Methods and Applications
• Foundations and modelling of dynamic networks using Dynamic Graph Neural Networks: A survey
• Graph-Based Deep Learning for Medical Diagnosis and Analysis: Past, Present and Future
• DeeperGCN: All You Need to Train Deeper GCNs
• Deep-learning Architecture for Short-term Passenger Flow Forecasting in Urban Rail Transit
• Learning Semantic Segmentation of Large-Scale Point Clouds with Random Sampling
• Design Space for Graph Neural Networks
• A Note on Over-Smoothing for Graph Neural Networks
• Towards Deeper Graph Neural Networks with Differentiable Group Normalization
• Parameter-Efficient Person Re-identification in the 3D Space
• On Representation Knowledge Distillation for Graph Neural Networks
• Graph Neural Ordinary Differential Equations
• PointASNL: Robust Point Clouds Processing using Nonlocal Neural Networks with Adaptive Sampling
• Graph-less Neural Networks: Teaching Old MLPs New Tricks via Distillation
• Tracklets Predicting Based Adaptive Graph Tracking
• Bridging the Gap between Spatial and Spectral Domains: A Survey on Graph Neural Networks
• Not only Look, but also Listen: Learning Multimodal Violence Detection under Weak Supervision
• A Closer Look at Local Aggregation Operators in Point Cloud Analysis
• Tackling the Challenges in Scene Graph Generation with Local-to-Global Interactions
• Attention-based Graph ResNet for Motor Intent Detection from Raw EEG signals
• Peeking into occluded joints: A novel framework for crowd pose estimation
• Advancing Molecular Machine Learning Representations with Stereoelectronics-Infused Molecular Graphs
• SGAS: Sequential Greedy Architecture Search
• Revisiting Graph Convolutional Network on Semi-Supervised Node Classification from an Optimization Perspective
• Room Classification on Floor Plan Graphs using Graph Neural Networks
• Training Matters: Unlocking Potentials of Deeper Graph Convolutional Neural Networks
• Recognizing Predictive Substructures with Subgraph Information Bottleneck
• Self-supervised Feature Learning by Cross-modality and Cross-view Correspondences
• Multi-task Graph Convolutional Neural Network for Calcification Morphology and Distribution Analysis in Mammograms
• GN-Transformer: Fusing Sequence and Graph Representation for Improved Code Summarization
• A Neural Rendering Framework for Free-Viewpoint Relighting
• Adversarial Graph Disentanglement
• HighwayGraph: Modelling Long-distance Node Relations for Improving General Graph Neural Network
• Graph Highway Networks
• LEReg: Empower Graph Neural Networks with Local Energy Regularization
• Structure-Aware Label Smoothing for Graph Neural Networks
• Segmentation of EM showers for neutrino experiments with deep graph neural networks
• Going Deeper with Lean Point Networks
• Fea2Fea: Exploring Structural Feature Correlations via Graph Neural Networks
• Knowledge Embedding Based Graph Convolutional Network
• Deep Representation Learning For Multimodal Brain Networks
• Graph-Based Tri-Attention Network for Answer Ranking in CQA
• How semantic and geometric information mutually reinforce each other in ToF object localization
• SN-Graph: a Minimalist 3D Object Representation for Classification
• DyDiff-VAE: A Dynamic Variational Framework for Information Diffusion Prediction
• Network Representation Learning: From Traditional Feature Learning to Deep Learning
• Characters as Graphs: Recognizing Online Handwritten Chinese Characters via Spatial Graph Convolutional Network
• SStaGCN: Simplified stacking based graph convolutional networks