Graph Foundation Models: Concepts, Opportunities and Challenges

Abstract

Foundation models have emerged as critical components in a variety of artificial intelligence applications, and showcase significant success in natural language processing and several other domains. Meanwhile, the field of graph machine learning is witnessing a paradigm transition from shallow methods to more sophisticated deep learning approaches. The capabilities of foundation models in generalization and adaptation motivate graph machine learning researchers to discuss the potential of developing a new graph learning paradigm. This paradigm envisions models that are pre-trained on extensive graph data and can be adapted for various graph tasks. Despite this burgeoning interest, there is a noticeable lack of clear definitions and systematic analyses pertaining to this new domain. To this end, this article introduces the concept of Graph Foundation Models (GFMs), and offers an exhaustive explanation of their key characteristics and underlying technologies. We proceed to classify the existing work related to GFMs into three distinct categories, based on their dependence on graph neural networks and large language models. In addition to providing a thorough review of the current state of GFMs, this article also outlooks potential avenues for future research in this rapidly evolving domain.

This is the author's version of the accepted paper (not the IEEE-published version). Citation information: DOI 10.1109/TPAMI.2025.3548729. For access to the final edited and published article, please follow the link provided: https://ieeexplore.ieee.org/document/10915556

References in corpus (27)

• A Comprehensive Survey on Graph Neural Networks
• DeepWalk: Online Learning of Social Representations
• Pre-trained Models for Natural Language Processing: A Survey
• Graph Neural Network for Traffic Forecasting: A Survey
• Unifying Large Language Models and Knowledge Graphs: A Roadmap
• GCC: Graph Contrastive Coding for Graph Neural Network Pre-Training
• Graph Self-Supervised Learning: A Survey
• Recommender Systems in the Era of Large Language Models (LLMs)
• FedGNN: Federated Graph Neural Network for Privacy-Preserving Recommendation
• Towards artificial general intelligence via a multimodal foundation model
• Recipe for a General, Powerful, Scalable Graph Transformer
• Representation Learning with Large Language Models for Recommendation
• A Theoretical Analysis of Contrastive Unsupervised Representation Learning
• Affinity Attention Graph Neural Network for Weakly Supervised Semantic Segmentation
• Learning and Evaluating Graph Neural Network Explanations based on Counterfactual and Factual Reasoning
• GIT-Mol: A Multi-modal Large Language Model for Molecular Science with Graph, Image, and Text
• Augmenting Low-Resource Text Classification with Graph-Grounded Pre-training and Prompting
• Uncertainty Quantification of Sparse Travel Demand Prediction with Spatial-Temporal Graph Neural Networks
• All in One and One for All: A Simple yet Effective Method towards Cross-domain Graph Pretraining
• Federated Heterogeneous Graph Neural Network for Privacy-preserving Recommendation
• IGB: Addressing The Gaps In Labeling, Features, Heterogeneity, and Size of Public Graph Datasets for Deep Learning Research
• GraphControl: Adding Conditional Control to Universal Graph Pre-trained Models for Graph Domain Transfer Learning
• Can we Soft Prompt LLMs for Graph Learning Tasks?
• Can Large Language Models Improve the Adversarial Robustness of Graph Neural Networks?
• Inductive Graph Alignment Prompt: Bridging the Gap between Graph Pre-training and Inductive Fine-tuning From Spectral Perspective
• Endowing Pre-trained Graph Models with Provable Fairness
• Voucher Abuse Detection with Prompt-based Fine-tuning on Graph Neural Networks

Cited by in corpus (4)

• A Survey on Self-Supervised Graph Foundation Models: Knowledge-Based Perspective
• From Anchors to Answers: A Novel Node Tokenizer for Integrating Graph Structure into Large Language Models
• A Survey on GNN-based Link Prediction: Techniques, Applications, and Challenges
• Knowledge Graphs Meet Graph Neural Networks: A Comprehensive Survey