Gated Graph Recurrent Neural Networks

Abstract

Graph processes exhibit a temporal structure determined by the sequence index and and a spatial structure determined by the graph support. To learn from graph processes, an information processing architecture must then be able to exploit both underlying structures. We introduce Graph Recurrent Neural Networks (GRNNs) as a general learning framework that achieves this goal by leveraging the notion of a recurrent hidden state together with graph signal processing (GSP). In the GRNN, the number of learnable parameters is independent of the length of the sequence and of the size of the graph, guaranteeing scalability. We prove that GRNNs are permutation equivariant and that they are stable to perturbations of the underlying graph support. To address the problem of vanishing gradients, we also put forward gated GRNNs with three different gating mechanisms: time, node and edge gates. In numerical experiments involving both synthetic and real datasets, time-gated GRNNs are shown to improve upon GRNNs in problems with long term dependencies, while node and edge gates help encode long range dependencies present in the graph. The numerical results also show that GRNNs outperform GNNs and RNNs, highlighting the importance of taking both the temporal and graph structures of a graph process into account.

Under review at IEEE TSP

References in corpus (10)

• On the difficulty of training Recurrent Neural Networks
• Discrete Signal Processing on Graphs
• Foundations and modelling of dynamic networks using Dynamic Graph Neural Networks: A survey
• Stability Properties of Graph Neural Networks
• LanczosNet: Multi-Scale Deep Graph Convolutional Networks
• Invariance-Preserving Localized Activation Functions for Graph Neural Networks
• Stability of Graph Scattering Transforms
• Controllability of Bandlimited Graph Processes Over Random Time Varying Graphs
• Estimating the epidemic risk using non-uniformly sampled contact data
• EvoNet: A Neural Network for Predicting the Evolution of Dynamic Graphs

Cited by in corpus (11)

• Attention-based graph neural networks: a survey
• HUNTER: AI based Holistic Resource Management for Sustainable Cloud Computing
• Disambiguated Node Classification with Graph Neural Networks
• Deep Representations for Time-varying Brain Datasets
• Input State Stability of Gated Graph Neural Networks
• Auto-Configured Explainable Graph Neural Networks for Multi-Site Pollution Prediction
• Assessment of Spatio-Temporal Predictors in the Presence of Missing and Heterogeneous Data
• Scalable Perception-Action-Communication Loops with Convolutional and Graph Neural Networks
• A Gated Graph Neural Network Approach to Fast-Convergent Dynamic Average Estimation
• Graph Neural Networks for Distributed Linear-Quadratic Control
• Graph Convolutional Memory using Topological Priors