A Survey on Self-Supervised Graph Foundation Models: Knowledge-Based Perspective
arXiv:2403.16137 · doi:10.1109/TKDE.2025.3568147
Abstract
Graph self-supervised learning (SSL) is now a go-to method for pre-training graph foundation models (GFMs). There is a wide variety of knowledge patterns embedded in the graph data, such as node properties and clusters, which are crucial to learning generalized representations for GFMs. However, existing surveys of GFMs have several shortcomings: they lack comprehensiveness regarding the most recent progress, have unclear categorization of self-supervised methods, and take a limited architecture-based perspective that is restricted to only certain types of graph models. As the ultimate goal of GFMs is to learn generalized graph knowledge, we provide a comprehensive survey of self-supervised GFMs from a novel knowledge-based perspective. We propose a knowledge-based taxonomy, which categorizes self-supervised graph models by the specific graph knowledge utilized. Our taxonomy consists of microscopic (nodes, links, etc.), mesoscopic (context, clusters, etc.), and macroscopic knowledge (global structure, manifolds, etc.). It covers a total of 9 knowledge categories and more than 25 pretext tasks for pre-training GFMs, as well as various downstream task generalization strategies. Such a knowledge-based taxonomy allows us to re-examine graph models based on new architectures more clearly, such as graph language models, as well as provide more in-depth insights for constructing GFMs.
Accepted by TKDE; full version (27 pages, 9 figures)
References in corpus (28)
- Fast unfolding of communities in large networks
- Survey of Hallucination in Natural Language Generation
- GCC: Graph Contrastive Coding for Graph Neural Network Pre-Training
- AM-GCN: Adaptive Multi-channel Graph Convolutional Networks
- SimGRACE: A Simple Framework for Graph Contrastive Learning without Data Augmentation
- Adaptive Graph Encoder for Attributed Graph Embedding
- Graph Barlow Twins: A self-supervised representation learning framework for graphs
- COSTA: Covariance-Preserving Feature Augmentation for Graph Contrastive Learning
- SelfKG: Self-Supervised Entity Alignment in Knowledge Graphs
- A Survey of Large Language Models for Graphs
- Augmenting Low-Resource Text Classification with Graph-Grounded Pre-training and Prompting
- Dual Space Graph Contrastive Learning
- Graph Communal Contrastive Learning
- HetGPT: Harnessing the Power of Prompt Tuning in Pre-Trained Heterogeneous Graph Neural Networks
- Generating Counterfactual Hard Negative Samples for Graph Contrastive Learning
- Graph Foundation Models: Concepts, Opportunities and Challenges
- Can we Soft Prompt LLMs for Graph Learning Tasks?
- GraphControl: Adding Conditional Control to Universal Graph Pre-trained Models for Graph Domain Transfer Learning
- Inductive Graph Alignment Prompt: Bridging the Gap between Graph Pre-training and Inductive Fine-tuning From Spectral Perspective
- Deep Contrastive Graph Learning with Clustering-Oriented Guidance
- Can Large Language Models Improve the Adversarial Robustness of Graph Neural Networks?
- Pre-Training and Prompting for Few-Shot Node Classification on Text-Attributed Graphs
- Graph Contrastive Learning with Cohesive Subgraph Awareness
- CARL-G: Clustering-Accelerated Representation Learning on Graphs
- Endowing Pre-trained Graph Models with Provable Fairness
- Mask-GVAE: Blind Denoising Graphs via Partition
- Balancing Graph Embedding Smoothness in Self-Supervised Learning via Information-Theoretic Decomposition
- DAGPrompT: Pushing the Limits of Graph Prompting with a Distribution-aware Graph Prompt Tuning Approach