From task structures to world models: What do LLMs know?

Abstract

In what sense does a large language model have knowledge? The answer to this question extends beyond the capabilities of a particular AI system, and challenges our assumptions about the nature of knowledge and intelligence. We answer by granting LLMs "instrumental knowledge"; knowledge defined by a certain set of abilities. We then ask how such knowledge is related to the more ordinary, "worldly" knowledge exhibited by human agents, and explore this in terms of the degree to which instrumental knowledge can be said to incorporate the structured world models of cognitive science. We discuss ways LLMs could recover degrees of worldly knowledge, and suggest such recovery will be governed by an implicit, resource-rational tradeoff between world models and task demands.

References in corpus (15)

• Training language models to follow instructions with human feedback
• LLaMA: Open and Efficient Foundation Language Models
• LoRA: Low-Rank Adaptation of Large Language Models
• Sparks of Artificial General Intelligence: Early experiments with GPT-4
• A tutorial introduction to the minimum description length principle
• People construct simplified mental representations to plan
• Eight Things to Know about Large Language Models
• Meaning without reference in large language models
• Emergent World Representations: Exploring a Sequence Model Trained on a Synthetic Task
• Modeling human intuitions about liquid flow with particle-based simulation
• From Word Models to World Models: Translating from Natural Language to the Probabilistic Language of Thought
• Physical problem solving: Joint planning with symbolic, geometric, and dynamic constraints
• TASRA: a Taxonomy and Analysis of Societal-Scale Risks from AI
• Emergent Representations of Program Semantics in Language Models Trained on Programs
• Tracr: Compiled Transformers as a Laboratory for Interpretability

Cited by in corpus (6)

• Six Fallacies in Substituting Large Language Models for Human Participants
• Transforming Agency. On the mode of existence of Large Language Models
• Chatting with Bots: AI, Speech Acts, and the Edge of Assertion
• What Do Large Language Models Know? Tacit Knowledge as a Potential Causal-Explanatory Structure
• Can structural correspondences ground real world representational content in Large Language Models?
• How LLMs Might Think