A Survey of Large Language Models for Graphs

Abstract

Graphs are an essential data structure utilized to represent relationships in real-world scenarios. Prior research has established that Graph Neural Networks (GNNs) deliver impressive outcomes in graph-centric tasks, such as link prediction and node classification. Despite these advancements, challenges like data sparsity and limited generalization capabilities continue to persist. Recently, Large Language Models (LLMs) have gained attention in natural language processing. They excel in language comprehension and summarization. Integrating LLMs with graph learning techniques has attracted interest as a way to enhance performance in graph learning tasks. In this survey, we conduct an in-depth review of the latest state-of-the-art LLMs applied in graph learning and introduce a novel taxonomy to categorize existing methods based on their framework design. We detail four unique designs: i) GNNs as Prefix, ii) LLMs as Prefix, iii) LLMs-Graphs Integration, and iv) LLMs-Only, highlighting key methodologies within each category. We explore the strengths and limitations of each framework, and emphasize potential avenues for future research, including overcoming current integration challenges between LLMs and graph learning techniques, and venturing into new application areas. This survey aims to serve as a valuable resource for researchers and practitioners eager to leverage large language models in graph learning, and to inspire continued progress in this dynamic field. We consistently maintain the related open-source materials at \url{https://github.com/HKUDS/Awesome-LLM4Graph-Papers}.

arXiv commentPublished as a KDD'24 survey paper

References in corpus (11)

• A Comprehensive Survey on Graph Neural Networks
• Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer
• A Survey on Large Language Model based Autonomous Agents
• Unifying Large Language Models and Knowledge Graphs: A Roadmap
• Representation Learning with Large Language Models for Recommendation
• Disentangled Contrastive Collaborative Filtering
• Graph Transformer for Recommendation
• GIT-Mol: A Multi-modal Large Language Model for Molecular Science with Graph, Image, and Text
• NodeFormer: A Scalable Graph Structure Learning Transformer for Node Classification
• Augmenting Low-Resource Text Classification with Graph-Grounded Pre-training and Prompting
• How Far Can Camels Go? Exploring the State of Instruction Tuning on Open Resources

Cited by in corpus (5)

• A Survey on Self-Supervised Graph Foundation Models: Knowledge-Based Perspective
• Advancing Tool-Augmented Large Language Models via Meta-Verification and Reflection Learning
• A Survey on Learning from Graphs with Heterophily: Recent Advances and Future Directions
• Graph Repairs with Large Language Models: An Empirical Study
• From Anchors to Answers: A Novel Node Tokenizer for Integrating Graph Structure into Large Language Models