Graph of Thoughts: Solving Elaborate Problems with Large Language Models

Abstract

We introduce Graph of Thoughts (GoT): a framework that advances prompting capabilities in large language models (LLMs) beyond those offered by paradigms such as Chain-of-Thought or Tree of Thoughts (ToT). The key idea and primary advantage of GoT is the ability to model the information generated by an LLM as an arbitrary graph, where units of information ("LLM thoughts") are vertices, and edges correspond to dependencies between these vertices. This approach enables combining arbitrary LLM thoughts into synergistic outcomes, distilling the essence of whole networks of thoughts, or enhancing thoughts using feedback loops. We illustrate that GoT offers advantages over state of the art on different tasks, for example increasing the quality of sorting by 62% over ToT, while simultaneously reducing costs by >31%. We ensure that GoT is extensible with new thought transformations and thus can be used to spearhead new prompting schemes. This work brings the LLM reasoning closer to human thinking or brain mechanisms such as recurrence, both of which form complex networks.

References in corpus (32)

• Chain-of-Thought Prompting Elicits Reasoning in Large Language Models
• LLaMA: Open and Efficient Foundation Language Models
• Geometric deep learning: going beyond Euclidean data
• Llama 2: Open Foundation and Fine-Tuned Chat Models
• PaLM: Scaling Language Modeling with Pathways
• Sparks of Artificial General Intelligence: Early experiments with GPT-4
• Self-Consistency Improves Chain of Thought Reasoning in Language Models
• ReAct: Synergizing Reasoning and Acting in Language Models
• Tree of Thoughts: Deliberate Problem Solving with Large Language Models
• Large Language Models Are Human-Level Prompt Engineers
• Representation Learning on Graphs: Methods and Applications
• Reflexion: Language Agents with Verbal Reinforcement Learning
• Self-Refine: Iterative Refinement with Self-Feedback
• Inner Monologue: Embodied Reasoning through Planning with Language Models
• Language Models as Zero-Shot Planners: Extracting Actionable Knowledge for Embodied Agents
• STaR: Bootstrapping Reasoning With Reasoning
• Selection-Inference: Exploiting Large Language Models for Interpretable Logical Reasoning
• Complexity-Based Prompting for Multi-Step Reasoning
• Foundation Models for Decision Making: Problems, Methods, and Opportunities
• Describe, Explain, Plan and Select: Interactive Planning with Large Language Models Enables Open-World Multi-Task Agents
• Large Language Model Guided Tree-of-Thought
• AI Chains: Transparent and Controllable Human-AI Interaction by Chaining Large Language Model Prompts
• REFINER: Reasoning Feedback on Intermediate Representations
• Beyond Chain-of-Thought, Effective Graph-of-Thought Reasoning in Language Models
• Solving Math Word Problems via Cooperative Reasoning induced Language Models
• Parallel and Distributed Graph Neural Networks: An In-Depth Concurrency Analysis
• Planning with Large Language Models for Code Generation
• The Graph Database Interface: Scaling Online Transactional and Analytical Graph Workloads to Hundreds of Thousands of Cores
• Skeleton-of-Thought: Prompting LLMs for Efficient Parallel Generation
• Self-Evaluation Guided Beam Search for Reasoning
• GraphMineSuite: Enabling High-Performance and Programmable Graph Mining Algorithms with Set Algebra
• Neural Graph Databases

Cited by in corpus (34)

• A Survey on Large Language Model based Autonomous Agents
• Unleashing the potential of prompt engineering for large language models
• Survey on Large Language Model-Enhanced Reinforcement Learning: Concept, Taxonomy, and Methods
• Generative AI for Self-Adaptive Systems: State of the Art and Research Roadmap
• On the Prospects of Incorporating Large Language Models (LLMs) in Automated Planning and Scheduling (APS)
• Correctness Comparison of ChatGPT-4, Gemini, Claude-3, and Copilot for Spatial Tasks
• From Screens to Scenes: A Survey of Embodied AI in Healthcare
• LightVA: Lightweight Visual Analytics with LLM Agent-Based Task Planning and Execution
• Aligning Large Language Models for Enhancing Psychiatric Interviews Through Symptom Delineation and Summarization: Pilot Study
• Integrating Graphs with Large Language Models: Methods and Prospects
• A Survey on Large Language Model-based Agents for Statistics and Data Science
• Beyond Accuracy: Investigating Error Types in GPT-4 Responses to USMLE Questions
• Computational Argumentation-based Chatbots: a Survey
• A Survey on Retrieval And Structuring Augmented Generation with Large Language Models
• pathfinder: A Semantic Framework for Literature Review and Knowledge Discovery in Astronomy
• Moderating New Waves of Online Hate with Chain-of-Thought Reasoning in Large Language Models
• Evaluating GPT- and Reasoning-based Large Language Models on Physics Olympiad Problems: Surpassing Human Performance and Implications for Educational Assessment
• LLM-Mediated Domain-Specific Voice Agents: The Case of TextileBot
• Demystifying Chains, Trees, and Graphs of Thoughts
• Large Language Models to the Rescue: Reducing the Complexity in Scientific Workflow Development Using ChatGPT
• NeuroSync: Intent-Aware Code-Based Problem Solving via Direct LLM Understanding Modification
• Web Archives Metadata Generation with GPT-4o: Challenges and Insights
• Thought Graph: Generating Thought Process for Biological Reasoning
• PromptSet: A Programmer's Prompting Dataset
• The Few-shot Dilemma: Over-prompting Large Language Models
• Improving Public Service Chatbot Design and Civic Impact: Investigation of Citizens' Perceptions of a Metro City 311 Chatbot
• LaTCoder: Converting Webpage Design to Code with Layout-as-Thought
• STARec: An Efficient Agent Framework for Recommender Systems via Autonomous Deliberate Reasoning
• PerfDojo: Automated ML Library Generation for Heterogeneous Architectures
• Efficient Serving of LLM Applications with Probabilistic Demand Modeling
• MMUEChange: A Generalized LLM Agent Framework for Intelligent Multi-Modal Urban Environment Change Analysis
• Adaptive Domain Modeling with Language Models: A Multi-Agent Approach to Task Planning
• Serve Programs, Not Prompts
• Imitation Game for Adversarial Disillusion with Chain-of-Thought Reasoning in Generative AI