Continuous-Depth Neural Models for Dynamic Graph Prediction
arXiv:2106.11581
Abstract
We introduce the framework of continuous-depth graph neural networks (GNNs). Neural graph differential equations (Neural GDEs) are formalized as the counterpart to GNNs where the input-output relationship is determined by a continuum of GNN layers, blending discrete topological structures and differential equations. The proposed framework is shown to be compatible with static GNN models and is extended to dynamic and stochastic settings through hybrid dynamical system theory. Here, Neural GDEs improve performance by exploiting the underlying dynamics geometry, further introducing the ability to accommodate irregularly sampled data. Results prove the effectiveness of the proposed models across applications, such as traffic forecasting or prediction in genetic regulatory networks.
arXiv commentExtended version of the workshop paper "Graph Neural Ordinary Differential Equations". arXiv admin note: substantial text overlap with arXiv:1911.07532
References in corpus (27)
- Adam: A Method for Stochastic Optimization
- Semi-Supervised Classification with Graph Convolutional Networks
- The Emerging Field of Signal Processing on Graphs: Extending High-Dimensional Data Analysis to Networks and Other Irregular Domains
- Learning Phrase Representations using RNN Encoder-Decoder for Statistical Machine Translation
- Spectral Networks and Locally Connected Networks on Graphs
- SGDR: Stochastic Gradient Descent with Warm Restarts
- Discrete Signal Processing on Graphs
- Simplifying Graph Convolutional Networks
- Hamiltonian Neural Networks
- Learning Deep Generative Models of Graphs
- Graph networks as learnable physics engines for inference and control
- Exact Combinatorial Optimization with Graph Convolutional Neural Networks
- Latent ODEs for Irregularly-Sampled Time Series
- Scalable Gradients for Stochastic Differential Equations
- FFJORD: Free-form Continuous Dynamics for Scalable Reversible Generative Models
- Neural Jump Stochastic Differential Equations
- ODEVAE: Deep generative second order ODEs with Bayesian neural networks
- Dissecting Neural ODEs
- Recurrent Neural Networks for Time Series Forecasting
- Hamiltonian Graph Networks with ODE Integrators
- Augmented Neural ODEs
- Continuous Graph Neural Networks
- Riemannian Continuous Normalizing Flows
- Neural Manifold Ordinary Differential Equations
- Model-based Reinforcement Learning for Semi-Markov Decision Processes with Neural ODEs
- G2SAT: Learning to Generate SAT Formulas
- TorchDyn: A Neural Differential Equations Library