Towards a Psychology of Machines: Large Language Models Predict Human Memory

Abstract

Large language models (LLMs), such as ChatGPT, have shown remarkable abilities in natural language processing, opening new avenues in psychological research. This study explores whether LLMs can predict human memory performance in tasks involving garden-path sentences and contextual information. In the first part, we used ChatGPT to rate the relatedness and memorability of garden-path sentences preceded by either fitting or unfitting contexts. In the second part, human participants read the same sentences, rated their relatedness, and completed a surprise memory test. The results demonstrated that ChatGPT's relatedness ratings closely matched those of the human participants, and its memorability ratings effectively predicted human memory performance. Both LLM and human data revealed that higher relatedness in the unfitting context condition was associated with better memory performance, aligning with probabilistic frameworks of context-dependent learning. These findings suggest that LLMs, despite lacking human-like memory mechanisms, can model aspects of human cognition and serve as valuable tools in psychological research. We propose the field of machine psychology to explore this interplay between human cognition and artificial intelligence, offering a bidirectional approach where LLMs can both benefit from and contribute to our understanding of human cognitive processes.

arXiv comment34 pages, 3 figures, 2 tables

References in corpus (8)

• ChatGPT Outperforms Crowd-Workers for Text-Annotation Tasks
• Using cognitive psychology to understand GPT-3
• A Comprehensive Overview of Large Language Models
• Large Language Models as Simulated Economic Agents: What Can We Learn from Homo Silicus?
• Is GPT a Computational Model of Emotion? Detailed Analysis
• Visual cognition in multimodal large language models
• Incremental Comprehension of Garden-Path Sentences by Large Language Models: Semantic Interpretation, Syntactic Re-Analysis, and Attention
• Evaluating Contextualized Representations of (Spanish) Ambiguous Words: A New Lexical Resource and Empirical Analysis