Graph Neural Ordinary Differential Equations

Abstract

We introduce the framework of continuous--depth graph neural networks (GNNs). Graph neural ordinary differential equations (GDEs) are formalized as the counterpart to GNNs where the input-output relationship is determined by a continuum of GNN layers, blending discrete topological structures and differential equations. The proposed framework is shown to be compatible with various static and autoregressive GNN models. Results prove general effectiveness of GDEs: in static settings they offer computational advantages by incorporating numerical methods in their forward pass; in dynamic settings, on the other hand, they are shown to improve performance by exploiting the geometry of the underlying dynamics.

arXiv commentAccepted [Spotlight] at the AAAI workshop DLGMA20. For the extended version, see "Continuous-Depth Neural Models for Dynamic Graph Prediction"

References in corpus (28)

• Adam: A Method for Stochastic Optimization
• Semi-Supervised Classification with Graph Convolutional Networks
• The Emerging Field of Signal Processing on Graphs: Extending High-Dimensional Data Analysis to Networks and Other Irregular Domains
• Learning Phrase Representations using RNN Encoder-Decoder for Statistical Machine Translation
• Spectral Networks and Locally Connected Networks on Graphs
• SGDR: Stochastic Gradient Descent with Warm Restarts
• Discrete Signal Processing on Graphs
• Representation Learning on Graphs with Jumping Knowledge Networks
• FractalNet: Ultra-Deep Neural Networks without Residuals
• GraphRNN: Generating Realistic Graphs with Deep Auto-regressive Models
• Learning Deep Generative Models of Graphs
• Combinatorial Optimization with Graph Convolutional Networks and Guided Tree Search
• Graph networks as learnable physics engines for inference and control
• Exact Combinatorial Optimization with Graph Convolutional Neural Networks
• Neural Relational Inference for Interacting Systems
• Latent ODEs for Irregularly-Sampled Time Series
• FFJORD: Free-form Continuous Dynamics for Scalable Reversible Generative Models
• DeepGCNs: Can GCNs Go as Deep as CNNs?
• Traffic Graph Convolutional Recurrent Neural Network: A Deep Learning Framework for Network-Scale Traffic Learning and Forecasting
• ODE${}^2$VAE: Deep generative second order ODEs with Bayesian neural networks
• Dissecting Neural ODEs
• ANODE: Unconditionally Accurate Memory-Efficient Gradients for Neural ODEs
• Hamiltonian Graph Networks with ODE Integrators
• Recurrent Neural Networks for Time Series Forecasting
• Augmented Neural ODEs
• GResNet: Graph Residual Network for Reviving Deep GNNs from Suspended Animation
• Continuous Graph Neural Networks
• Graph Normalizing Flows

Cited by in corpus (18)

• Dissecting Neural ODEs
• Nonlocal Kernel Network (NKN): a Stable and Resolution-Independent Deep Neural Network
• Evolutionary Preference Learning via Graph Nested GRU ODE for Session-based Recommendation
• Continuous-in-Depth Neural Networks
• Hypersolvers: Toward Fast Continuous-Depth Models
• Dissecting the Diffusion Process in Linear Graph Convolutional Networks
• Beltrami Flow and Neural Diffusion on Graphs
• Heavy Ball Neural Ordinary Differential Equations
• 3D-IDS: Doubly Disentangled Dynamic Intrusion Detection
• TorchDyn: A Neural Differential Equations Library
• Structural Inference of Networked Dynamical Systems with Universal Differential Equations
• Stable Neural Flows
• Learning continuous-time PDEs from sparse data with graph neural networks
• Data-Centric Engineering: integrating simulation, machine learning and statistics. Challenges and Opportunities
• Towards Scale-Invariant Graph-related Problem Solving by Iterative Homogeneous Graph Neural Networks
• Learning Continuous Network Emerging Dynamics from Scarce Observations via Data-Adaptive Stochastic Processes
• Reversible and irreversible bracket-based dynamics for deep graph neural networks
• Robust and Scalable SDE Learning: A Functional Perspective