Understanding the Representation Power of Graph Neural Networks in Learning Graph Topology

Abstract

To deepen our understanding of graph neural networks, we investigate the representation power of Graph Convolutional Networks (GCN) through the looking glass of graph moments, a key property of graph topology encoding path of various lengths. We find that GCNs are rather restrictive in learning graph moments. Without careful design, GCNs can fail miserably even with multiple layers and nonlinear activation functions. We analyze theoretically the expressiveness of GCNs, concluding a modular GCN design, using different propagation rules with residual connections could significantly improve the performance of GCN. We demonstrate that such modular designs are capable of distinguishing graphs from different graph generation models for surprisingly small graphs, a notoriously difficult problem in network science. Our investigation suggests that, depth is much more influential than width, with deeper GCNs being more capable of learning higher order graph moments. Additionally, combining GCN modules with different propagation rules is critical to the representation power of GCNs.

Cited by in corpus (26)

• Graph Neural Networks: A Review of Methods and Applications
• Principal Neighbourhood Aggregation for Graph Nets
• Metrics for Graph Comparison: A Practitioner's Guide
• Graph Neural Networks: Taxonomy, Advances and Trends
• Tackling Over-Smoothing for General Graph Convolutional Networks
• Revisiting Over-smoothing in Deep GCNs
• Generalization and Representational Limits of Graph Neural Networks
• Are Powerful Graph Neural Nets Necessary? A Dissection on Graph Classification
• Continuous Graph Neural Networks
• Graph Neural Networks: Methods, Applications, and Opportunities
• Understanding the Message Passing in Graph Neural Networks via Power Iteration Clustering
• What graph neural networks cannot learn: depth vs width
• Predicting cognitive scores with graph neural networks through sample selection learning
• How hard is to distinguish graphs with graph neural networks?
• Spatial Aggregation and Temporal Convolution Networks for Real-time Kriging
• Contrastive and Generative Graph Convolutional Networks for Graph-based Semi-Supervised Learning
• RelEx: A Model-Agnostic Relational Model Explainer
• Utilising Graph Machine Learning within Drug Discovery and Development
• H-VGRAE: A Hierarchical Stochastic Spatial-Temporal Embedding Method for Robust Anomaly Detection in Dynamic Networks
• Multi-Scale 2D Temporal Adjacent Networks for Moment Localization with Natural Language
• Multi-Resolution Graph Neural Network for Large-Scale Pointcloud Segmentation
• Breaking the Expressive Bottlenecks of Graph Neural Networks
• SoGCN: Second-Order Graph Convolutional Networks
• On the Universality of Graph Neural Networks on Large Random Graphs
• Multi-Level Attention Pooling for Graph Neural Networks: Unifying Graph Representations with Multiple Localities
• Eigen-GNN: A Graph Structure Preserving Plug-in for GNNs