Residual Gated Graph ConvNets

Abstract

Graph-structured data such as social networks, functional brain networks, gene regulatory networks, communications networks have brought the interest in generalizing deep learning techniques to graph domains. In this paper, we are interested to design neural networks for graphs with variable length in order to solve learning problems such as vertex classification, graph classification, graph regression, and graph generative tasks. Most existing works have focused on recurrent neural networks (RNNs) to learn meaningful representations of graphs, and more recently new convolutional neural networks (ConvNets) have been introduced. In this work, we want to compare rigorously these two fundamental families of architectures to solve graph learning tasks. We review existing graph RNN and ConvNet architectures, and propose natural extension of LSTM and ConvNet to graphs with arbitrary size. Then, we design a set of analytically controlled experiments on two basic graph problems, i.e. subgraph matching and graph clustering, to test the different architectures. Numerical results show that the proposed graph ConvNets are 3-17% more accurate and 1.5-4x faster than graph RNNs. Graph ConvNets are also 36% more accurate than variational (non-learning) techniques. Finally, the most effective graph ConvNet architecture uses gated edges and residuality. Residuality plays an essential role to learn multi-layer architectures as they provide a 10% gain of performance.

References in corpus (5)

• Empirical Evaluation of Gated Recurrent Neural Networks on Sequence Modeling
• Neural Message Passing for Quantum Chemistry
• Deep Convolutional Networks on Graph-Structured Data
• Geometric deep learning on graphs and manifolds using mixture model CNNs
• Encoding Sentences with Graph Convolutional Networks for Semantic Role Labeling

Cited by in corpus (64)

• A Generalization of Transformer Networks to Graphs
• Improving Graph Neural Network Expressivity via Subgraph Isomorphism Counting
• PhyCRNet: Physics-informed Convolutional-Recurrent Network for Solving Spatiotemporal PDEs
• Spatio-temporal graph neural networks for multi-site PV power forecasting
• Principal Neighbourhood Aggregation for Graph Nets
• Benchmarking Graph Neural Networks
• An Efficient Graph Convolutional Network Technique for the Travelling Salesman Problem
• Graph Neural Networks: Taxonomy, Advances and Trends
• Learning Multi-dimensional Edge Feature-based AU Relation Graph for Facial Action Unit Recognition
• Do Transformers Really Perform Bad for Graph Representation?
• Global Self-Attention as a Replacement for Graph Convolution
• A Comparison of Neural Network Architectures for Data-Driven Reduced-Order Modeling
• Rethinking Graph Transformers with Spectral Attention
• The Evolution of Distributed Systems for Graph Neural Networks and their Origin in Graph Processing and Deep Learning: A Survey
• SCENE: Reasoning about Traffic Scenes using Heterogeneous Graph Neural Networks
• A Two-Step Graph Convolutional Decoder for Molecule Generation
• Contrastive Graph Pooling for Explainable Classification of Brain Networks
• NeuroLKH: Combining Deep Learning Model with Lin-Kernighan-Helsgaun Heuristic for Solving the Traveling Salesman Problem
• Graph Neural Networks for Graph Drawing
• Hierarchical Inter-Message Passing for Learning on Molecular Graphs
• Neural Large Neighborhood Search for the Capacitated Vehicle Routing Problem
• Graph Neural Networks with Learnable Structural and Positional Representations
• Lifelong Learning of Graph Neural Networks for Open-World Node Classification
• A community-powered search of machine learning strategy space to find NMR property prediction models
• Lifelong Learning on Evolving Graphs Under the Constraints of Imbalanced Classes and New Classes
• Dorylus: Affordable, Scalable, and Accurate GNN Training with Distributed CPU Servers and Serverless Threads
• Towards better Interpretable and Generalizable AD detection using Collective Artificial Intelligence
• A deep learning model for data-driven discovery of functional connectivity
• Bayesian Graph Convolutional Neural Networks using Node Copying
• SSFG: Stochastically Scaling Features and Gradients for Regularizing Graph Convolutional Networks
• Identification of vortex in unstructured mesh with graph neural networks
• Learning Graph Normalization for Graph Neural Networks
• Global Attention Improves Graph Networks Generalization
• How to Evaluate Machine Learning Approaches for Combinatorial Optimization: Application to the Travelling Salesman Problem
• Directional Graph Networks
• Bayesian Graph Convolutional Neural Networks Using Non-Parametric Graph Learning
• Rethinking Graph Neural Architecture Search from Message-passing
• Lymph Node Graph Neural Networks for Cancer Metastasis Prediction
• Weisfeiler and Lehman Go Paths: Learning Topological Features via Path Complexes
• Weisfeiler and Lehman Go Cellular: CW Networks
• A pipeline for fair comparison of graph neural networks in node classification tasks
• Policy-GNN: Aggregation Optimization for Graph Neural Networks
• Boosting Column Generation with Graph Neural Networks for Joint Rider Trip Planning and Crew Shift Scheduling
• A benchmark study on reliable molecular supervised learning via Bayesian learning
• Tree Structure-Aware Graph Representation Learning via Integrated Hierarchical Aggregation and Relational Metric Learning
• On Self-Distilling Graph Neural Network
• Extracting real estate values of rental apartment floor plans using graph convolutional networks
• CPR-GCN: Conditional Partial-Residual Graph Convolutional Network in Automated Anatomical Labeling of Coronary Arteries
• Learning Graph Representation of Person-specific Cognitive Processes from Audio-visual Behaviours for Automatic Personality Recognition
• From Spectrum Wavelet to Vertex Propagation: Graph Convolutional Networks Based on Taylor Approximation
• Amortized Probabilistic Detection of Communities in Graphs
• SoGCN: Second-Order Graph Convolutional Networks
• Feature Correlation Aggregation: on the Path to Better Graph Neural Networks
• Deep Constraint-based Propagation in Graph Neural Networks
• Topological Graph Neural Networks
• Hybrid Focal and Full-Range Attention Based Graph Transformers
• ProDyn0: Inferring calponin homology domain stretching behavior using graph neural networks
• Experiments with graph convolutional networks for solving the vertex $p$-center problem
• Enhancing Social Relation Inference with Concise Interaction Graph and Discriminative Scene Representation
• Graph Feature Gating Networks
• Edge-featured Graph Neural Architecture Search
• Meta-Aggregator: Learning to Aggregate for 1-bit Graph Neural Networks
• A Graph Neural Network Approach for Scalable and Dynamic IP Similarity in Enterprise Networks
• An Experimental Study of the Transferability of Spectral Graph Networks