PDE-GCN: Novel Architectures for Graph Neural Networks Motivated by Partial Differential Equations

Abstract

Graph neural networks are increasingly becoming the go-to approach in various fields such as computer vision, computational biology and chemistry, where data are naturally explained by graphs. However, unlike traditional convolutional neural networks, deep graph networks do not necessarily yield better performance than shallow graph networks. This behavior usually stems from the over-smoothing phenomenon. In this work, we propose a family of architectures to control this behavior by design. Our networks are motivated by numerical methods for solving Partial Differential Equations (PDEs) on manifolds, and as such, their behavior can be explained by similar analysis. Moreover, as we demonstrate using an extensive set of experiments, our PDE-motivated networks can generalize and be effective for various types of problems from different fields. Our architectures obtain better or on par with the current state-of-the-art results for problems that are typically approached using different architectures.

arXiv commentNeurIPS 2021

References in corpus (18)

• Adam: A Method for Stochastic Optimization
• PyTorch: An Imperative Style, High-Performance Deep Learning Library
• Semi-Supervised Classification with Graph Convolutional Networks
• Inductive Representation Learning on Large Graphs
• Neural Message Passing for Quantum Chemistry
• Spectral Networks and Locally Connected Networks on Graphs
• Cluster-GCN: An Efficient Algorithm for Training Deep and Large Graph Convolutional Networks
• MeshCNN: A Network with an Edge
• DropEdge: Towards Deep Graph Convolutional Networks on Node Classification
• PDE-Net 2.0: Learning PDEs from Data with A Numeric-Symbolic Hybrid Deep Network
• DeepInf: Social Influence Prediction with Deep Learning
• Learning data driven discretizations for partial differential equations
• Simple and Deep Graph Convolutional Networks
• Dynamic Graph CNN for Learning on Point Clouds
• Learning shape correspondence with anisotropic convolutional neural networks
• Geom-GCN: Geometric Graph Convolutional Networks
• IMEXnet: A Forward Stable Deep Neural Network
• LeanConvNets: Low-cost Yet Effective Convolutional Neural Networks

Cited by in corpus (2)

• Structural Inference of Networked Dynamical Systems with Universal Differential Equations
• Quantized Convolutional Neural Networks Through the Lens of Partial Differential Equations