Reversible and irreversible bracket-based dynamics for deep graph neural networks

Abstract

Recent works have shown that physics-inspired architectures allow the training of deep graph neural networks (GNNs) without oversmoothing. The role of these physics is unclear, however, with successful examples of both reversible (e.g., Hamiltonian) and irreversible (e.g., diffusion) phenomena producing comparable results despite diametrically opposed mechanisms, and further complications arising due to empirical departures from mathematical theory. This work presents a series of novel GNN architectures based upon structure-preserving bracket-based dynamical systems, which are provably guaranteed to either conserve energy or generate positive dissipation with increasing depth. It is shown that the theoretically principled framework employed here allows for inherently explainable constructions, which contextualize departures from theory in current architectures and better elucidate the roles of reversibility and irreversibility in network performance.

References in corpus (29)

• Adam: A Method for Stochastic Optimization
• Discovering governing equations from data: Sparse identification of nonlinear dynamical systems
• Hidden Physics Models: Machine Learning of Nonlinear Partial Differential Equations
• Pitfalls of Graph Neural Network Evaluation
• OpenAI Gym
• Geometric Deep Learning: Grids, Groups, Graphs, Geodesics, and Gauges
• Quantifying total uncertainty in physics-informed neural networks for solving forward and inverse stochastic problems
• Hamiltonian Neural Networks
• Adversarial Uncertainty Quantification in Physics-Informed Neural Networks
• Multiscale modeling of inelastic materials with Thermodynamics-based Artificial Neural Networks (TANN)
• Symplectic ODE-Net: Learning Hamiltonian Dynamics with Control
• A Note on Over-Smoothing for Graph Neural Networks
• Structure-preserving neural networks
• Towards Deeper Graph Neural Networks with Differentiable Group Normalization
• GFINNs: GENERIC Formalism Informed Neural Networks for Deterministic and Stochastic Dynamical Systems
• Graph Neural Ordinary Differential Equations
• Thermodynamics-informed graph neural networks
• Implicit Graph Neural Networks
• Port-Hamiltonian Neural Networks for Learning Explicit Time-Dependent Dynamical Systems
• Continuous Graph Neural Networks
• Dissipative SymODEN: Encoding Hamiltonian Dynamics with Dissipation and Control into Deep Learning
• Data-driven Prediction of General Hamiltonian Dynamics via Learning Exactly-Symplectic Maps
• Deconstructing the Inductive Biases of Hamiltonian Neural Networks
• Understanding convolution on graphs via energies
• Energetically Consistent Model Reduction for Metriplectic Systems
• Graph-Coupled Oscillator Networks
• Structure-preserving Sparse Identification of Nonlinear Dynamics for Data-driven Modeling
• Enforcing exact physics in scientific machine learning: a data-driven exterior calculus on graphs
• GREAD: Graph Neural Reaction-Diffusion Networks