Generalization and Representational Limits of Graph Neural Networks

Abstract

We address two fundamental questions about graph neural networks (GNNs). First, we prove that several important graph properties cannot be computed by GNNs that rely entirely on local information. Such GNNs include the standard message passing models, and more powerful spatial variants that exploit local graph structure (e.g., via relative orientation of messages, or local port ordering) to distinguish neighbors of each node. Our treatment includes a novel graph-theoretic formalism. Second, we provide the first data dependent generalization bounds for message passing GNNs. This analysis explicitly accounts for the local permutation invariance of GNNs. Our bounds are much tighter than existing VC-dimension based guarantees for GNNs, and are comparable to Rademacher bounds for recurrent neural networks.

References in corpus (14)

• Inductive Representation Learning on Large Graphs
• Neural Message Passing for Quantum Chemistry
• Interaction Networks for Learning about Objects, Relations and Physics
• Graph Transformer Networks
• Discriminative Embeddings of Latent Variable Models for Structured Data
• A Survey on Graph Kernels
• Learning Multimodal Graph-to-Graph Translation for Molecular Optimization
• Measuring abstract reasoning in neural networks
• Supervised Community Detection with Line Graph Neural Networks
• Invariant and Equivariant Graph Networks
• Position-aware Graph Neural Networks
• Understanding the Representation Power of Graph Neural Networks in Learning Graph Topology
• Efficient Probabilistic Logic Reasoning with Graph Neural Networks
• On Generalization Bounds of a Family of Recurrent Neural Networks

Cited by in corpus (25)

• Improving Graph Neural Network Expressivity via Subgraph Isomorphism Counting
• Principal Neighbourhood Aggregation for Graph Nets
• Graph Neural Networks: Taxonomy, Advances and Trends
• Machine Learning on Graphs: A Model and Comprehensive Taxonomy
• On the Bottleneck of Graph Neural Networks and its Practical Implications
• A Survey on The Expressive Power of Graph Neural Networks
• Deep Learning Techniques for In-Crop Weed Identification: A Review
• Learning the Travelling Salesperson Problem Requires Rethinking Generalization
• On the design space between molecular mechanics and machine learning force fields
• Can Graph Neural Networks Count Substructures?
• Expressive Power of Invariant and Equivariant Graph Neural Networks
• Building powerful and equivariant graph neural networks with structural message-passing
• Pointer Graph Networks
• How hard is to distinguish graphs with graph neural networks?
• Erdos Goes Neural: an Unsupervised Learning Framework for Combinatorial Optimization on Graphs
• A PAC-Bayesian Approach to Generalization Bounds for Graph Neural Networks
• Global Attention Improves Graph Networks Generalization
• From Local Structures to Size Generalization in Graph Neural Networks
• Graph Convolution for Semi-Supervised Classification: Improved Linear Separability and Out-of-Distribution Generalization
• On Graph Neural Networks versus Graph-Augmented MLPs
• Optimization and Generalization Analysis of Transduction through Gradient Boosting and Application to Multi-scale Graph Neural Networks
• Understanding Deep Architectures with Reasoning Layer
• Improving Subgraph Matching by Combining Algorithms and Graph Neural Networks
• Multi-fidelity Stability for Graph Representation Learning
• Explanation from Specification