GraphGLOW: Universal and Generalizable Structure Learning for Graph Neural Networks
arXiv:2306.11264 · doi:10.1145/3580305.3599373
Abstract
Graph structure learning is a well-established problem that aims at optimizing graph structures adaptive to specific graph datasets to help message passing neural networks (i.e., GNNs) to yield effective and robust node embeddings. However, the common limitation of existing models lies in the underlying \textit{closed-world assumption}: the testing graph is the same as the training graph. This premise requires independently training the structure learning model from scratch for each graph dataset, which leads to prohibitive computation costs and potential risks for serious over-fitting. To mitigate these issues, this paper explores a new direction that moves forward to learn a universal structure learning model that can generalize across graph datasets in an open world. We first introduce the mathematical definition of this novel problem setting, and describe the model formulation from a probabilistic data-generative aspect. Then we devise a general framework that coordinates a single graph-shared structure learner and multiple graph-specific GNNs to capture the generalizable patterns of optimal message-passing topology across datasets. The well-trained structure learner can directly produce adaptive structures for unseen target graphs without any fine-tuning. Across diverse datasets and various challenging cross-graph generalization protocols, our experiments show that even without training on target graphs, the proposed model i) significantly outperforms expressive GNNs trained on input (non-optimized) topology, and ii) surprisingly performs on par with state-of-the-art models that independently optimize adaptive structures for specific target graphs, with notably orders-of-magnitude acceleration for training on the target graph.
arXiv commentPublished as a conference paper at KDD 2023
References in corpus (26)
- Show, Attend and Tell: Neural Image Caption Generation with Visual Attention
- Geometric deep learning: going beyond Euclidean data
- Categorical Reparameterization with Gumbel-Softmax
- Graph Neural Network for Traffic Forecasting: A Survey
- Representation Learning on Graphs with Jumping Knowledge Networks
- Social Structure of Facebook Networks
- Domain Generalization via Invariant Feature Representation
- Learning to Simulate Complex Physics with Graph Networks
- Predict then Propagate: Graph Neural Networks meet Personalized PageRank
- Iterative Deep Graph Learning for Graph Neural Networks: Better and Robust Node Embeddings
- Causal Attention for Interpretable and Generalizable Graph Classification
- Learning Discrete Structures for Graph Neural Networks
- Graph Information Bottleneck
- NodeFormer: A Scalable Graph Structure Learning Transformer for Node Classification
- Is Homophily a Necessity for Graph Neural Networks?
- Handling Distribution Shifts on Graphs: An Invariance Perspective
- Subgroup Generalization and Fairness of Graph Neural Networks
- DIFFormer: Scalable (Graph) Transformers Induced by Energy Constrained Diffusion
- New Benchmarks for Learning on Non-Homophilous Graphs
- Graph Neural Networks are Inherently Good Generalizers: Insights by Bridging GNNs and MLPs
- Predicting the Silent Majority on Graphs: Knowledge Transferable Graph Neural Network
- Training Robust Graph Neural Networks with Topology Adaptive Edge Dropping
- Towards Open-World Recommendation: An Inductive Model-based Collaborative Filtering Approach
- Towards Open-World Feature Extrapolation: An Inductive Graph Learning Approach
- Energy-based Out-of-Distribution Detection for Graph Neural Networks
- Geometric Knowledge Distillation: Topology Compression for Graph Neural Networks