A Fair Comparison of Graph Neural Networks for Graph Classification

Abstract

Experimental reproducibility and replicability are critical topics in machine learning. Authors have often raised concerns about their lack in scientific publications to improve the quality of the field. Recently, the graph representation learning field has attracted the attention of a wide research community, which resulted in a large stream of works. As such, several Graph Neural Network models have been developed to effectively tackle graph classification. However, experimental procedures often lack rigorousness and are hardly reproducible. Motivated by this, we provide an overview of common practices that should be avoided to fairly compare with the state of the art. To counter this troubling trend, we ran more than 47000 experiments in a controlled and uniform framework to re-evaluate five popular models across nine common benchmarks. Moreover, by comparing GNNs with structure-agnostic baselines we provide convincing evidence that, on some datasets, structural information has not been exploited yet. We believe that this work can contribute to the development of the graph learning field, by providing a much needed grounding for rigorous evaluations of graph classification models.

arXiv commentExtended version of the paper published at the International Conference on Learning Representations (ICLR), 2020. Additional results are shown in the appendix

References in corpus (5)

• Neural Message Passing for Quantum Chemistry
• Fast Graph Representation Learning with PyTorch Geometric
• Pitfalls of Graph Neural Network Evaluation
• Dynamic Edge-Conditioned Filters in Convolutional Neural Networks on Graphs
• Contextual Graph Markov Model: A Deep and Generative Approach to Graph Processing

Cited by in corpus (41)

• Graph Neural Networks: A Review of Methods and Applications
• A Gentle Introduction to Deep Learning for Graphs
• TUDataset: A collection of benchmark datasets for learning with graphs
• Benchmarking Graph Neural Networks
• Design Space for Graph Neural Networks
• Graph Kernels: A Survey
• On the Bottleneck of Graph Neural Networks and its Practical Implications
• Deep Graph Matching and Searching for Semantic Code Retrieval
• Accurate Learning of Graph Representations with Graph Multiset Pooling
• Understanding the Performance of Knowledge Graph Embeddings in Drug Discovery
• Rethinking pooling in graph neural networks
• Stealing Links from Graph Neural Networks
• Interpreting and Unifying Graph Neural Networks with An Optimization Framework
• Explaining Deep Graph Networks with Molecular Counterfactuals
• Edge Representation Learning with Hypergraphs
• Residual Network and Embedding Usage: New Tricks of Node Classification with Graph Convolutional Networks
• Re-evaluating Word Mover's Distance
• On Positional and Structural Node Features for Graph Neural Networks on Non-attributed Graphs
• Hybrid Micro/Macro Level Convolution for Heterogeneous Graph Learning
• Graph Neural Networks with Parallel Neighborhood Aggregations for Graph Classification
• Adaptive-Step Graph Meta-Learner for Few-Shot Graph Classification
• CogDL: A Comprehensive Library for Graph Deep Learning
• A Hyperbolic-to-Hyperbolic Graph Convolutional Network
• MxPool: Multiplex Pooling for Hierarchical Graph Representation Learning
• Tensor Decompositions in Deep Learning
• Ego-based Entropy Measures for Structural Representations
• A Novel Higher-order Weisfeiler-Lehman Graph Convolution
• Graph Cross Networks with Vertex Infomax Pooling
• Wasserstein Embedding for Graph Learning
• Graph-based State Representation for Deep Reinforcement Learning
• Learnable Structural Semantic Readout for Graph Classification
• Evaluating Modules in Graph Contrastive Learning
• Graph Random Neural Features for Distance-Preserving Graph Representations
• AKE-GNN: Effective Graph Learning with Adaptive Knowledge Exchange
• Catastrophic Forgetting in Deep Graph Networks: an Introductory Benchmark for Graph Classification
• Connecting Graph Convolutional Networks and Graph-Regularized PCA
• Data-Driven Self-Supervised Graph Representation Learning
• GraphGen-Redux: a Fast and Lightweight Recurrent Model for labeled Graph Generation
• A GNN Based Approach to LTL Model Checking
• Event Detection on Dynamic Graphs
• Graphs for deep learning representations