Open Graph Benchmark: Datasets for Machine Learning on Graphs

Abstract

We present the Open Graph Benchmark (OGB), a diverse set of challenging and realistic benchmark datasets to facilitate scalable, robust, and reproducible graph machine learning (ML) research. OGB datasets are large-scale, encompass multiple important graph ML tasks, and cover a diverse range of domains, ranging from social and information networks to biological networks, molecular graphs, source code ASTs, and knowledge graphs. For each dataset, we provide a unified evaluation protocol using meaningful application-specific data splits and evaluation metrics. In addition to building the datasets, we also perform extensive benchmark experiments for each dataset. Our experiments suggest that OGB datasets present significant challenges of scalability to large-scale graphs and out-of-distribution generalization under realistic data splits, indicating fruitful opportunities for future research. Finally, OGB provides an automated end-to-end graph ML pipeline that simplifies and standardizes the process of graph data loading, experimental setup, and model evaluation. OGB will be regularly updated and welcomes inputs from the community. OGB datasets as well as data loaders, evaluation scripts, baseline code, and leaderboards are publicly available at https://ogb.stanford.edu .

arXiv commentFix dataset bug in ogbg-code

References in corpus (26)

• Distributed Representations of Words and Phrases and their Compositionality
• PyTorch: An Imperative Style, High-Performance Deep Learning Library
• Neural Message Passing for Quantum Chemistry
• Geometric deep learning: going beyond Euclidean data
• MXNet: A Flexible and Efficient Machine Learning Library for Heterogeneous Distributed Systems
• A Review of Relational Machine Learning for Knowledge Graphs
• Cluster-GCN: An Efficient Algorithm for Training Deep and Large Graph Convolutional Networks
• Fast Graph Representation Learning with PyTorch Geometric
• Simplifying Graph Convolutional Networks
• OpenML: networked science in machine learning
• Graph Convolutional Matrix Completion
• Representation Learning on Graphs with Jumping Knowledge Networks
• Variational Graph Auto-Encoders
• RotatE: Knowledge Graph Embedding by Relational Rotation in Complex Space
• DropEdge: Towards Deep Graph Convolutional Networks on Node Classification
• Pitfalls of Graph Neural Network Evaluation
• PyTorch-BigGraph: A Large-scale Graph Embedding System
• Benchmarking Graph Neural Networks
• Strategies for Pre-training Graph Neural Networks
• Position-aware Graph Neural Networks
• Learning Graph-Level Representation for Drug Discovery
• DGL-KE: Training Knowledge Graph Embeddings at Scale
• OpenML-Python: an extensible Python API for OpenML
• Understanding Isomorphism Bias in Graph Data Sets
• Evaluating Logical Generalization in Graph Neural Networks
• Dimensional Reweighting Graph Convolutional Networks

Cited by in corpus (157)

• Graph Contrastive Learning with Augmentations
• Deep Graph Library: A Graph-Centric, Highly-Performant Package for Graph Neural Networks
• TUDataset: A collection of benchmark datasets for learning with graphs
• DeeperGCN: All You Need to Train Deeper GCNs
• Principal Neighbourhood Aggregation for Graph Nets
• Towards Out-Of-Distribution Generalization: A Survey
• Benchmarking Graph Neural Networks
• Iterative Deep Graph Learning for Graph Neural Networks: Better and Robust Node Embeddings
• SIGN: Scalable Inception Graph Neural Networks
• Beyond Homophily in Graph Neural Networks: Current Limitations and Effective Designs
• Graph Neural Networks: Taxonomy, Advances and Trends
• Do Transformers Really Perform Bad for Graph Representation?
• How to Find Your Friendly Neighborhood: Graph Attention Design with Self-Supervision
• Combining Label Propagation and Simple Models Out-performs Graph Neural Networks
• Neural Bellman-Ford Networks: A General Graph Neural Network Framework for Link Prediction
• Masked Label Prediction: Unified Message Passing Model for Semi-Supervised Classification
• Machine Learning on Graphs: A Model and Comprehensive Taxonomy
• GraphNorm: A Principled Approach to Accelerating Graph Neural Network Training
• Rethinking Graph Transformers with Spectral Attention
• The Evolution of Distributed Systems for Graph Neural Networks and their Origin in Graph Processing and Deep Learning: A Survey
• Anomaly Detection in Dynamic Graphs: A Comprehensive Survey
• GOOD-D: On Unsupervised Graph Out-Of-Distribution Detection
• Explaining the Explainers in Graph Neural Networks: a Comparative Study
• Approximate Graph Propagation
• Learning Strong Graph Neural Networks with Weak Information
• Graph-less Neural Networks: Teaching Old MLPs New Tricks via Distillation
• Mask and Reason: Pre-Training Knowledge Graph Transformers for Complex Logical Queries
• How Attentive are Graph Attention Networks?
• Dirichlet Energy Constrained Learning for Deep Graph Neural Networks
• Hierarchical Inter-Message Passing for Learning on Molecular Graphs
• Be Confident! Towards Trustworthy Graph Neural Networks via Confidence Calibration
• Multi-hop Attention Graph Neural Network
• Wiki-CS: A Wikipedia-Based Benchmark for Graph Neural Networks
• Path Integral Based Convolution and Pooling for Graph Neural Networks
• Nested Graph Neural Networks
• Directed Acyclic Graph Neural Networks
• Graph Neural Networks: Methods, Applications, and Opportunities
• Can Graph Neural Networks Count Substructures?
• Bag of Tricks for Node Classification with Graph Neural Networks
• Subgroup Generalization and Fairness of Graph Neural Networks
• A Unified Lottery Ticket Hypothesis for Graph Neural Networks
• From Stars to Subgraphs: Uplifting Any GNN with Local Structure Awareness
• Scalable Graph Neural Networks for Heterogeneous Graphs
• Graph Neural Networks with Learnable Structural and Positional Representations
• New Benchmarks for Learning on Non-Homophilous Graphs
• Understanding and Resolving Performance Degradation in Graph Convolutional Networks
• SLAPS: Self-Supervision Improves Structure Learning for Graph Neural Networks
• DistDGL: Distributed Graph Neural Network Training for Billion-Scale Graphs
• Shift-Robust GNNs: Overcoming the Limitations of Localized Graph Training Data
• Lifelong Learning on Evolving Graphs Under the Constraints of Imbalanced Classes and New Classes
• Accelerating Training and Inference of Graph Neural Networks with Fast Sampling and Pipelining
• Scalable and Adaptive Graph Neural Networks with Self-Label-Enhanced training
• Graph Condensation for Graph Neural Networks
• GNNShap: Scalable and Accurate GNN Explanation using Shapley Values
• Can Large Language Models Improve the Adversarial Robustness of Graph Neural Networks?
• Improving Graph Neural Networks with Simple Architecture Design
• PairRE: Knowledge Graph Embeddings via Paired Relation Vectors
• Motif-Driven Contrastive Learning of Graph Representations
• Evaluating Deep Graph Neural Networks
• Topology-aware Embedding Memory for Continual Learning on Expanding Networks
• Beltrami Flow and Neural Diffusion on Graphs
• Large-scale graph representation learning with very deep GNNs and self-supervision
• Acc-SpMM: Accelerating General-purpose Sparse Matrix-Matrix Multiplication with GPU Tensor Cores
• Graph Neural Networks Inspired by Classical Iterative Algorithms
• Deep Graph Neural Networks with Shallow Subgraph Samplers
• Serpens: A High Bandwidth Memory Based Accelerator for General-Purpose Sparse Matrix-Vector Multiplication
• SUREL+: Moving from Walks to Sets for Scalable Subgraph-based Graph Representation Learning
• VQ-GNN: A Universal Framework to Scale up Graph Neural Networks using Vector Quantization
• Graph Meta Learning via Local Subgraphs
• Global Attention Improves Graph Networks Generalization
• Directional Graph Networks
• On the Unreasonable Effectiveness of Feature propagation in Learning on Graphs with Missing Node Features
• Learning to Represent Programs with Heterogeneous Graphs
• Graph Posterior Network: Bayesian Predictive Uncertainty for Node Classification
• Revisiting Graph Convolutional Network on Semi-Supervised Node Classification from an Optimization Perspective
• From Local Structures to Size Generalization in Graph Neural Networks
• Generalizing Graph Neural Networks on Out-Of-Distribution Graphs
• Secure Deep Graph Generation with Link Differential Privacy
• Learning Scalable Structural Representations for Link Prediction with Bloom Signatures
• DotHash: Estimating Set Similarity Metrics for Link Prediction and Document Deduplication
• Network In Graph Neural Network
• R-GSN: The Relation-based Graph Similar Network for Heterogeneous Graph
• Ranking over Regression for Bayesian Optimization and Molecule Selection
• LCS Graph Kernel Based on Wasserstein Distance in Longest Common Subsequence Metric Space
• Calibrating and Improving Graph Contrastive Learning
• Weisfeiler and Lehman Go Paths: Learning Topological Features via Path Complexes
• Graph Networks with Spectral Message Passing
• Graph convolutions that can finally model local structure
• Lipschitz Normalization for Self-Attention Layers with Application to Graph Neural Networks
• Why Propagate Alone? Parallel Use of Labels and Features on Graphs
• Accelerating SpMM Kernel with Cache-First Edge Sampling for Graph Neural Networks
• Spectral Geometric Matrix Completion
• Open Bandit Dataset and Pipeline: Towards Realistic and Reproducible Off-Policy Evaluation
• A pipeline for fair comparison of graph neural networks in node classification tasks
• Highly Accurate Disease Diagnosis and Highly Reproducible Biomarker Identification with PathFormer
• Wasserstein Embedding for Graph Learning
• Leiden-Fusion Partitioning Method for Effective Distributed Training of Graph Embeddings
• Boosting Multitask Learning on Graphs through Higher-Order Task Affinities
• Molecular Graph Representation Learning via Structural Similarity Information
• Robustness of Graph Neural Networks at Scale
• Node-Level Differentially Private Graph Neural Networks
• Graph Inference Representation: Learning Graph Positional Embeddings with Anchor Path Encoding
• PyTorch-Direct: Enabling GPU Centric Data Access for Very Large Graph Neural Network Training with Irregular Accesses
• Enhance Information Propagation for Graph Neural Network by Heterogeneous Aggregations
• A Biased Graph Neural Network Sampler with Near-Optimal Regret
• Better Schedules for Low Precision Training of Deep Neural Networks
• Graph Classification by Mixture of Diverse Experts
• Breaking the Expressive Bottlenecks of Graph Neural Networks
• Implicit SVD for Graph Representation Learning
• Releasing Graph Neural Networks with Differential Privacy Guarantees
• Message Passing in Graph Convolution Networks via Adaptive Filter Banks
• CHILI: Chemically-Informed Large-scale Inorganic Nanomaterials Dataset for Advancing Graph Machine Learning
• The Split Matters: Flat Minima Methods for Improving the Performance of GNNs
• Tackling the Local Bias in Federated Graph Learning
• Embedding Graphs on Grassmann Manifold
• Graph Belief Propagation Networks
• Polynomial Selection in Spectral Graph Neural Networks: An Error-Sum of Function Slices Approach
• SoGCN: Second-Order Graph Convolutional Networks
• Should Graph Neural Networks Use Features, Edges, Or Both?
• Learning Conjoint Attentions for Graph Neural Nets
• Learning Parametrised Graph Shift Operators
• Blind Demixing of Diffused Graph Signals
• Inductive learning for product assortment graph completion
• Analyzing the Performance of Graph Neural Networks with Pipe Parallelism
• Explaining Deep Learning Representations by Tracing the Training Process
• Global Neighbor Sampling for Mixed CPU-GPU Training on Giant Graphs
• On Local Aggregation in Heterophilic Graphs
• Connecting Graph Convolutional Networks and Graph-Regularized PCA
• Explainable Biomedical Recommendations via Reinforcement Learning Reasoning on Knowledge Graphs
• Knowledge Graph Completion using Structural and Textual Embeddings
• Graph Self-Supervised Learning with Learnable Structural and Positional Encodings
• GABO: Graph Augmentations with Bi-level Optimization
• Optimizing Memory Efficiency of Graph Neural Networks on Edge Computing Platforms
• Multipath Graph Convolutional Neural Networks
• Density of States Graph Kernels
• Improving Graph Property Prediction with Generalized Readout Functions
• Latent Tree Decomposition Parsers for AMR-to-Text Generation
• An Experimental Study of the Transferability of Spectral Graph Networks
• Diversified Multiscale Graph Learning with Graph Self-Correction
• Graph Context Encoder: Graph Feature Inpainting for Graph Generation and Self-supervised Pretraining
• CUQ-GNN: Committee-based Graph Uncertainty Quantification using Posterior Networks
• Does your graph need a confidence boost? Convergent boosted smoothing on graphs with tabular node features
• Fast Graph Learning with Unique Optimal Solutions
• Sequential Aggregation and Rematerialization: Distributed Full-batch Training of Graph Neural Networks on Large Graphs
• Revisiting SVD to generate powerful Node Embeddings for Recommendation Systems
• Memory-Associated Differential Learning
• Scalable Consistency Training for Graph Neural Networks via Self-Ensemble Self-Distillation
• Partition-wise Graph Filtering: A Unified Perspective Through the Lens of Graph Coarsening
• RaWaNet: Enriching Graph Neural Network Input via Random Walks on Graphs
• Steganography of Complex Networks
• Implicit vs Unfolded Graph Neural Networks
• Edge-similarity-aware Graph Neural Networks
• SAS: A Simple, Accurate and Scalable Node Classification Algorithm
• Joint Use of Node Attributes and Proximity for Semi-Supervised Classification on Graphs
• Graph Neural Network Training with Data Tiering
• Scaling up graph homomorphism for classification via sampling
• Graphs for deep learning representations