Building powerful and equivariant graph neural networks with structural message-passing

Abstract

Message-passing has proved to be an effective way to design graph neural networks, as it is able to leverage both permutation equivariance and an inductive bias towards learning local structures in order to achieve good generalization. However, current message-passing architectures have a limited representation power and fail to learn basic topological properties of graphs. We address this problem and propose a powerful and equivariant message-passing framework based on two ideas: first, we propagate a one-hot encoding of the nodes, in addition to the features, in order to learn a local context matrix around each node. This matrix contains rich local information about both features and topology and can eventually be pooled to build node representations. Second, we propose methods for the parametrization of the message and update functions that ensure permutation equivariance. Having a representation that is independent of the specific choice of the one-hot encoding permits inductive reasoning and leads to better generalization properties. Experimentally, our model can predict various graph topological properties on synthetic data more accurately than previous methods and achieves state-of-the-art results on molecular graph regression on the ZINC dataset.

arXiv commentPresented at Neurips 2020

References in corpus (26)

• Neural Message Passing for Quantum Chemistry
• SchNet - a deep learning architecture for molecules and materials
• Graph Neural Networks: A Review of Methods and Applications
• Fast Graph Representation Learning with PyTorch Geometric
• Learning Combinatorial Optimization Algorithms over Graphs
• Deep learning for molecular design - a review of the state of the art
• Interaction Networks for Learning about Objects, Relations and Physics
• Combinatorial Optimization with Graph Convolutional Networks and Guided Tree Search
• Improving Graph Neural Network Expressivity via Subgraph Isomorphism Counting
• Principal Neighbourhood Aggregation for Graph Nets
• Benchmarking Graph Neural Networks
• An Efficient Graph Convolutional Network Technique for the Travelling Salesman Problem
• Universal Invariant and Equivariant Graph Neural Networks
• Invariant and Equivariant Graph Networks
• What Can Neural Networks Reason About?
• On the Bottleneck of Graph Neural Networks and its Practical Implications
• Covariant Compositional Networks For Learning Graphs
• A Survey on The Expressive Power of Graph Neural Networks
• Generalization and Representational Limits of Graph Neural Networks
• Weisfeiler and Leman go sparse: Towards scalable higher-order graph embeddings
• Hierarchical Inter-Message Passing for Learning on Molecular Graphs
• Can Graph Neural Networks Count Substructures?
• How hard is to distinguish graphs with graph neural networks?
• Let's Agree to Degree: Comparing Graph Convolutional Networks in the Message-Passing Framework
• Directional Graph Networks
• The Power of Graph Convolutional Networks to Distinguish Random Graph Models: Short Version

Cited by in corpus (8)

• Nested Graph Neural Networks
• Spherical Message Passing for 3D Graph Networks
• Improving the Expressiveness of $K$-hop Message-Passing GNNs by Injecting Contextualized Substructure Information
• MuGSI: Distilling GNNs with Multi-Granularity Structural Information for Graph Classification
• Counting Substructures with Higher-Order Graph Neural Networks: Possibility and Impossibility Results
• Breaking the Expressive Bottlenecks of Graph Neural Networks
• Graph Self-Supervised Learning with Learnable Structural and Positional Encodings
• Edge-featured Graph Neural Architecture Search