DeepGCNs: Making GCNs Go as Deep as CNNs

Abstract

Convolutional Neural Networks (CNNs) have been very successful at solving a variety of computer vision tasks such as object classification and detection, semantic segmentation, activity understanding, to name just a few. One key enabling factor for their great performance has been the ability to train very deep networks. Despite their huge success in many tasks, CNNs do not work well with non-Euclidean data, which is prevalent in many real-world applications. Graph Convolutional Networks (GCNs) offer an alternative that allows for non-Eucledian data input to a neural network. While GCNs already achieve encouraging results, they are currently limited to architectures with a relatively small number of layers, primarily due to vanishing gradients during training. This work transfers concepts such as residual/dense connections and dilated convolutions from CNNs to GCNs in order to successfully train very deep GCNs. We show the benefit of using deep GCNs (with as many as 112 layers) experimentally across various datasets and tasks. Specifically, we achieve very promising performance in part segmentation and semantic segmentation on point clouds and in node classification of protein functions across biological protein-protein interaction (PPI) graphs. We believe that the insights in this work will open avenues for future research on GCNs and their application to further tasks not explored in this paper. The source code for this work is available at https://github.com/lightaime/deep_gcns_torch and https://github.com/lightaime/deep_gcns for PyTorch and TensorFlow implementation respectively.

Accepted at TPAMI. This work is a journal extension of our ICCV'19 paper arXiv:1904.03751. The first three authors contributed equally

References in corpus (9)

• Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift
• Semi-Supervised Classification with Graph Convolutional Networks
• Rethinking Atrous Convolution for Semantic Image Segmentation
• Fast Graph Representation Learning with PyTorch Geometric
• Predicting multicellular function through multi-layer tissue networks
• DeeperGCN: All You Need to Train Deeper GCNs
• Benchmarking Graph Neural Networks
• Cross-Sentence N-ary Relation Extraction with Graph LSTMs
• Towards Deeper Graph Neural Networks with Differentiable Group Normalization

Cited by in corpus (23)

• DeeperGCN: All You Need to Train Deeper GCNs
• Benchmarking Graph Neural Networks
• Deep Learning-based 3D Point Cloud Classification: A Systematic Survey and Outlook
• PGCN: Pyramidal Graph Convolutional Network for EEG Emotion Recognition
• Enable Deep Learning on Mobile Devices: Methods, Systems, and Applications
• GraphCFC: A Directed Graph Based Cross-Modal Feature Complementation Approach for Multimodal Conversational Emotion Recognition
• GraphMFT: A Graph Network based Multimodal Fusion Technique for Emotion Recognition in Conversation
• ASSANet: An Anisotropic Separable Set Abstraction for Efficient Point Cloud Representation Learning
• PU-GCN: Point Cloud Upsampling using Graph Convolutional Networks
• A Hybrid Similarity-Aware Graph Neural Network with Transformer for Node Classification
• A Dual-Stream Recurrence-Attention Network With Global-Local Awareness for Emotion Recognition in Textual Dialog
• SGAS: Sequential Greedy Architecture Search
• A Graph-Based Method for Soccer Action Spotting Using Unsupervised Player Classification
• Grid-GCN for Fast and Scalable Point Cloud Learning
• Towards Representation Identical Privacy-Preserving Graph Neural Network via Split Learning
• LiDAR Loop Closure Detection using Semantic Graphs with Graph Attention Networks
• Fusion Flow-enhanced Graph Pooling Residual Networks for Unmanned Aerial Vehicles Surveillance in Day and Night Dual Visions
• HyperGALE: ASD Classification via Hypergraph Gated Attention with Learnable Hyperedges
• 3D Photo Stylization: Learning to Generate Stylized Novel Views from a Single Image
• Random Projection Forest Initialization for Graph Convolutional Networks
• SALA: Soft Assignment Local Aggregation for Parameter Efficient 3D Semantic Segmentation
• Exploring Deep 3D Spatial Encodings for Large-Scale 3D Scene Understanding
• GraphSeam: Supervised Graph Learning Framework for Semantic UV Mapping