Large Language Models as Psychological Simulators: A Methodological Guide

Abstract

Large language models (LLMs) offer emerging opportunities for psychological and behavioral research, but methodological guidance is lacking. This article provides a framework for using LLMs as psychological simulators across two primary applications: simulating roles and personas to explore diverse contexts, and serving as computational models to investigate cognitive processes. For simulation, we present methods for developing psychologically grounded personas that move beyond demographic categories, with strategies for validation against human data and use cases ranging from studying inaccessible populations to prototyping research instruments. For cognitive modeling, we synthesize emerging approaches for probing internal representations, methodological advances in causal interventions, and strategies for relating model behavior to human cognition. We address overarching challenges including prompt sensitivity, temporal limitations from training data cutoffs, and ethical considerations that extend beyond traditional human subjects review. Throughout, we emphasize the need for transparency about model capabilities and constraints. Together, this framework integrates emerging empirical evidence about LLM performance--including systematic biases, cultural limitations, and prompt brittleness--to help researchers wrangle these challenges and leverage the unique capabilities of LLMs in psychological research.

References in corpus (11)

• Using cognitive psychology to understand GPT-3
• Cultural Bias and Cultural Alignment of Large Language Models
• Techniques for supercharging academic writing with generative AI
• Large Language Models Portray Socially Subordinate Groups as More Homogeneous, Consistent with a Bias Observed in Humans
• Exploring the Frontiers of LLMs in Psychological Applications: A Comprehensive Review
• Six Fallacies in Substituting Large Language Models for Human Participants
• Large language models can replicate cross-cultural differences in personality
• Surveying the Dead Minds: Historical-Psychological Text Analysis with Contextualized Construct Representation (CCR) for Classical Chinese
• Large-scale moral machine experiment on large language models
• A validity-guided workflow for robust large language model research in psychology
• From Prompts to Constructs: A Dual-Validity Framework for Large Language Model Research in Psychology