Are we really making much progress? Revisiting, benchmarking, and refining heterogeneous graph neural networks

Abstract

Heterogeneous graph neural networks (HGNNs) have been blossoming in recent years, but the unique data processing and evaluation setups used by each work obstruct a full understanding of their advancements. In this work, we present a systematical reproduction of 12 recent HGNNs by using their official codes, datasets, settings, and hyperparameters, revealing surprising findings about the progress of HGNNs. We find that the simple homogeneous GNNs, e.g., GCN and GAT, are largely underestimated due to improper settings. GAT with proper inputs can generally match or outperform all existing HGNNs across various scenarios. To facilitate robust and reproducible HGNN research, we construct the Heterogeneous Graph Benchmark (HGB), consisting of 11 diverse datasets with three tasks. HGB standardizes the process of heterogeneous graph data splits, feature processing, and performance evaluation. Finally, we introduce a simple but very strong baseline Simple-HGN--which significantly outperforms all previous models on HGB--to accelerate the advancement of HGNNs in the future.

arXiv commentKDD 2021 research track

References in corpus (8)

• Semi-Supervised Classification with Graph Convolutional Networks
• BPR: Bayesian Personalized Ranking from Implicit Feedback
• Fast and Accurate Deep Network Learning by Exponential Linear Units (ELUs)
• KGAT: Knowledge Graph Attention Network for Recommendation
• MAGNN: Metapath Aggregated Graph Neural Network for Heterogeneous Graph Embedding
• Graph Transformer Networks
• Representation Learning for Attributed Multiplex Heterogeneous Network
• KRED: Knowledge-Aware Document Representation for News Recommendations

Cited by in corpus (7)

• Learning Multi-granularity User Intent Unit for Session-based Recommendation
• xFraud: Explainable Fraud Transaction Detection
• Homophily-oriented Heterogeneous Graph Rewiring
• Space4HGNN: A Novel, Modularized and Reproducible Platform to Evaluate Heterogeneous Graph Neural Network
• Activity-aware Human Mobility Prediction with Hierarchical Graph Attention Recurrent Network
• Heterogeneous Social Event Detection via Hyperbolic Graph Representations
• Micro-video Tagging via Jointly Modeling Social Influence and Tag Relation