Benchmarking Graph Neural Networks

Abstract

In the last few years, graph neural networks (GNNs) have become the standard toolkit for analyzing and learning from data on graphs. This emerging field has witnessed an extensive growth of promising techniques that have been applied with success to computer science, mathematics, biology, physics and chemistry. But for any successful field to become mainstream and reliable, benchmarks must be developed to quantify progress. This led us in March 2020 to release a benchmark framework that i) comprises of a diverse collection of mathematical and real-world graphs, ii) enables fair model comparison with the same parameter budget to identify key architectures, iii) has an open-source, easy-to-use and reproducible code infrastructure, and iv) is flexible for researchers to experiment with new theoretical ideas. As of December 2022, the GitHub repository has reached 2,000 stars and 380 forks, which demonstrates the utility of the proposed open-source framework through the wide usage by the GNN community. In this paper, we present an updated version of our benchmark with a concise presentation of the aforementioned framework characteristics, an additional medium-sized molecular dataset AQSOL, similar to the popular ZINC, but with a real-world measured chemical target, and discuss how this framework can be leveraged to explore new GNN designs and insights. As a proof of value of our benchmark, we study the case of graph positional encoding (PE) in GNNs, which was introduced with this benchmark and has since spurred interest of exploring more powerful PE for Transformers and GNNs in a robust experimental setting.

Benchmarking framework on GitHub at https://github.com/graphdeeplearning/benchmarking-gnns

References in corpus (19)

• PyTorch: An Imperative Style, High-Performance Deep Learning Library
• Fast Graph Representation Learning with PyTorch Geometric
• Interaction Networks for Learning about Objects, Relations and Physics
• Open Graph Benchmark: Datasets for Machine Learning on Graphs
• Learning to Simulate Complex Physics with Graph Networks
• Self-Attention Graph Pooling
• A Gentle Introduction to Deep Learning for Graphs
• Revisiting Graph Neural Networks: All We Have is Low-Pass Filters
• Fake News Detection on Social Media using Geometric Deep Learning
• SIGN: Scalable Inception Graph Neural Networks
• Chip Placement with Deep Reinforcement Learning
• A Fair Comparison of Graph Neural Networks for Graph Classification
• Machine Learning on Graphs: A Model and Comprehensive Taxonomy
• Position-aware Graph Neural Networks
• A Survey of Deep Learning for Scientific Discovery
• A Two-Step Graph Convolutional Decoder for Molecule Generation
• Learning Symbolic Physics with Graph Networks
• On Graph Classification Networks, Datasets and Baselines
• Redundancy-Free Computation Graphs for Graph Neural Networks

Cited by in corpus (113)

• Graph Contrastive Learning with Augmentations
• Learning Intents behind Interactions with Knowledge Graph for Recommendation
• Open Graph Benchmark: Datasets for Machine Learning on Graphs
• A Generalization of Transformer Networks to Graphs
• TUDataset: A collection of benchmark datasets for learning with graphs
• Improving Graph Neural Network Expressivity via Subgraph Isomorphism Counting
• Principal Neighbourhood Aggregation for Graph Nets
• Design Space for Graph Neural Networks
• Graph Neural Networks: Taxonomy, Advances and Trends
• Adversarial Graph Augmentation to Improve Graph Contrastive Learning
• Do Transformers Really Perform Bad for Graph Representation?
• A Survey on Graph-Based Deep Learning for Computational Histopathology
• Masked Label Prediction: Unified Message Passing Model for Semi-Supervised Classification
• Machine Learning on Graphs: A Model and Comprehensive Taxonomy
• Towards Deeper Graph Neural Networks with Differentiable Group Normalization
• GraphNorm: A Principled Approach to Accelerating Graph Neural Network Training
• Factorizable Graph Convolutional Networks
• Rethinking Graph Transformers with Spectral Attention
• Automated Machine Learning on Graphs: A Survey
• Accurate Learning of Graph Representations with Graph Multiset Pooling
• Understanding the Performance of Knowledge Graph Embeddings in Drug Discovery
• On Representation Knowledge Distillation for Graph Neural Networks
• Graph Kernels: State-of-the-Art and Future Challenges
• Rethinking pooling in graph neural networks
• Graph Clustering with Graph Neural Networks
• Beyond Low-pass Filtering: Graph Convolutional Networks with Automatic Filtering
• PGM-Explainer: Probabilistic Graphical Model Explanations for Graph Neural Networks
• How Attentive are Graph Attention Networks?
• Learning the Travelling Salesperson Problem Requires Rethinking Generalization
• Graph Neural Networks for Graph Drawing
• Stealing Links from Graph Neural Networks
• Hierarchical Inter-Message Passing for Learning on Molecular Graphs
• Wiki-CS: A Wikipedia-Based Benchmark for Graph Neural Networks
• Graph Neural Networks: Methods, Applications, and Opportunities
• Heterogeneous Network Representation Learning: A Unified Framework with Survey and Benchmark
• Finding Patient Zero: Learning Contagion Source with Graph Neural Networks
• Can Graph Neural Networks Count Substructures?
• Graph Deep Learning for Time Series Forecasting
• A Unified Lottery Ticket Hypothesis for Graph Neural Networks
• From Stars to Subgraphs: Uplifting Any GNN with Local Structure Awareness
• Graph Neural Networks with Learnable Structural and Positional Representations
• New Benchmarks for Learning on Non-Homophilous Graphs
• Lifelong Learning of Graph Neural Networks for Open-World Node Classification
• Expressive Power of Invariant and Equivariant Graph Neural Networks
• Visualizing Graph Neural Networks with CorGIE: Corresponding a Graph to Its Embedding
• Building powerful and equivariant graph neural networks with structural message-passing
• Degree-Quant: Quantization-Aware Training for Graph Neural Networks
• Pointer Graph Networks
• Edge Representation Learning with Hypergraphs
• Beyond Graph Neural Networks with Lifted Relational Neural Networks
• Spatial Aggregation and Temporal Convolution Networks for Real-time Kriging
• Erdos Goes Neural: an Unsupervised Learning Framework for Combinatorial Optimization on Graphs
• Large-scale graph representation learning with very deep GNNs and self-supervision
• Equivariant Subgraph Aggregation Networks
• Multiresolution Graph Transformers and Wavelet Positional Encoding for Learning Hierarchical Structures
• Learning Graph Normalization for Graph Neural Networks
• Geometric Scattering Attention Networks
• Evaluating Curriculum Learning Strategies in Neural Combinatorial Optimization
• Global Attention Improves Graph Networks Generalization
• Ego-GNNs: Exploiting Ego Structures in Graph Neural Networks
• HistoCartography: A Toolkit for Graph Analytics in Digital Pathology
• Directional Graph Networks
• Graph Convolutional Networks for traffic anomaly
• From Local Structures to Size Generalization in Graph Neural Networks
• You are AllSet: A Multiset Function Framework for Hypergraph Neural Networks
• Reconstructing nodal pressures in water distribution systems with graph neural networks
• MathNet: Haar-Like Wavelet Multiresolution-Analysis for Graph Representation and Learning
• Relational Graph Learning on Visual and Kinematics Embeddings for Accurate Gesture Recognition in Robotic Surgery
• Neural Sentence Ordering Based on Constraint Graphs
• Adversarial Attacks on Graph Classification via Bayesian Optimisation
• Lymph Node Graph Neural Networks for Cancer Metastasis Prediction
• Rethinking Graph Neural Architecture Search from Message-passing
• Open Bandit Dataset and Pipeline: Towards Realistic and Reproducible Off-Policy Evaluation
• CogDL: A Comprehensive Library for Graph Deep Learning
• GCN-SL: Graph Convolutional Networks with Structure Learning for Graphs under Heterophily
• A pipeline for fair comparison of graph neural networks in node classification tasks
• Pose-GNN : Camera Pose Estimation System Using Graph Neural Networks
• Weisfeiler and Lehman Go Cellular: CW Networks
• Maximum Entropy Weighted Independent Set Pooling for Graph Neural Networks
• A benchmark study on reliable molecular supervised learning via Bayesian learning
• Reconstruction for Powerful Graph Representations
• Evaluating Modules in Graph Contrastive Learning
• Structure-Aware Hard Negative Mining for Heterogeneous Graph Contrastive Learning
• Learning Graph Representation of Person-specific Cognitive Processes from Audio-visual Behaviours for Automatic Personality Recognition
• Parameter Prediction for Unseen Deep Architectures
• On Self-Distilling Graph Neural Network
• Breaking the Expressive Bottlenecks of Graph Neural Networks
• Efficient Colon Cancer Grading with Graph Neural Networks
• Should Graph Neural Networks Use Features, Edges, Or Both?
• Graph Computing for Financial Crime and Fraud Detection: Trends, Challenges and Outlook
• SoGCN: Second-Order Graph Convolutional Networks
• Message Passing in Graph Convolution Networks via Adaptive Filter Banks
• Amortized Probabilistic Detection of Communities in Graphs
• From Spectrum Wavelet to Vertex Propagation: Graph Convolutional Networks Based on Taylor Approximation
• Feature Correlation Aggregation: on the Path to Better Graph Neural Networks
• Partition and Code: learning how to compress graphs
• Topological Graph Neural Networks
• Deep Constraint-based Propagation in Graph Neural Networks
• Learning Conjoint Attentions for Graph Neural Nets
• On the Importance of Sampling in Training GCNs: Tighter Analysis and Variance Reduction
• Fea2Fea: Exploring Structural Feature Correlations via Graph Neural Networks
• Learning Parametrised Graph Shift Operators
• An Experimental Study of the Transferability of Spectral Graph Networks
• Scaling up graph homomorphism for classification via sampling
• Graph Neural Network for Hamiltonian-Based Material Property Prediction
• A Graph Neural Network Approach for Scalable and Dynamic IP Similarity in Enterprise Networks
• The Role of Isomorphism Classes in Multi-Relational Datasets
• LSP : Acceleration and Regularization of Graph Neural Networks via Locality Sensitive Pruning of Graphs
• Symmetry-driven graph neural networks
• Graph2Graph Learning with Conditional Autoregressive Models
• Edge-featured Graph Neural Architecture Search
• Edge but not Least: Cross-View Graph Pooling
• Meta-Aggregator: Learning to Aggregate for 1-bit Graph Neural Networks