Gated Graph Recurrent Neural Networks

Abstract

Graph processes exhibit a temporal structure determined by the sequence index and and a spatial structure determined by the graph support. To learn from graph processes, an information processing architecture must then be able to exploit both underlying structures. We introduce Graph Recurrent Neural Networks (GRNNs) as a general learning framework that achieves this goal by leveraging the notion of a recurrent hidden state together with graph signal processing (GSP). In the GRNN, the number of learnable parameters is independent of the length of the sequence and of the size of the graph, guaranteeing scalability. We prove that GRNNs are permutation equivariant and that they are stable to perturbations of the underlying graph support. To address the problem of vanishing gradients, we also put forward gated GRNNs with three different gating mechanisms: time, node and edge gates. In numerical experiments involving both synthetic and real datasets, time-gated GRNNs are shown to improve upon GRNNs in problems with long term dependencies, while node and edge gates help encode long range dependencies present in the graph. The numerical results also show that GRNNs outperform GNNs and RNNs, highlighting the importance of taking both the temporal and graph structures of a graph process into account.

arXiv commentUnder review at IEEE TSP

References in corpus (21)

• Adam: A Method for Stochastic Optimization
• The Emerging Field of Signal Processing on Graphs: Extending High-Dimensional Data Analysis to Networks and Other Irregular Domains
• On the difficulty of training Recurrent Neural Networks
• Spatio-Temporal Graph Convolutional Networks: A Deep Learning Framework for Traffic Forecasting
• Discrete Signal Processing on Graphs
• Contact patterns in a high school: a comparison between data collected using wearable sensors, contact diaries and friendship surveys
• Foundations and modelling of dynamic networks using Dynamic Graph Neural Networks: A survey
• Convolutional Neural Network Architectures for Signals Supported on Graphs
• Stationary signal processing on graphs
• Stationary Graph Processes and Spectral Estimation
• Stability Properties of Graph Neural Networks
• LanczosNet: Multi-Scale Deep Graph Convolutional Networks
• Lipschitz regularity of deep neural networks: analysis and efficient estimation
• Advances in Distributed Graph Filtering
• Invariance-Preserving Localized Activation Functions for Graph Neural Networks
• Accelerated filtering on graphs using Lanczos method
• Stability of Graph Scattering Transforms
• Controllability of Bandlimited Graph Processes Over Random Time Varying Graphs
• Estimating the epidemic risk using non-uniformly sampled contact data
• Ergodicity in Stationary Graph Processes: A Weak Law of Large Numbers
• EvoNet: A Neural Network for Predicting the Evolution of Dynamic Graphs

Cited by in corpus (11)

• Attention-based graph neural networks: a survey
• HUNTER: AI based Holistic Resource Management for Sustainable Cloud Computing
• Disambiguated Node Classification with Graph Neural Networks
• Deep Representations for Time-varying Brain Datasets
• Input State Stability of Gated Graph Neural Networks
• Auto-Configured Explainable Graph Neural Networks for Multi-Site Pollution Prediction
• Assessment of Spatio-Temporal Predictors in the Presence of Missing and Heterogeneous Data
• Scalable Perception-Action-Communication Loops with Convolutional and Graph Neural Networks
• A Gated Graph Neural Network Approach to Fast-Convergent Dynamic Average Estimation
• Graph Neural Networks for Distributed Linear-Quadratic Control
• Graph Convolutional Memory using Topological Priors