1 citations · 1 across the 7 of their papers we have counts for
8 papers · 1 filter
Developmental Trajectories of Situation Modeling and Mentalizing in Transformer Language Models
Pamela D. Rivière, Cameron Jones, Sean Trott
Recent work suggests that Large Language Models (LLMs) are sensitive to the belief states of agents described by text, as measured by the false belief task (FBT), yet persistent co…
How Open Must Language Models be to Enable Reliable Scientific Inference?
James A. Michaelov, Catherine Arnett, Tyler A. Chang +7
How does the extent to which a model is open or closed impact the scientific inferences that can be drawn from research that involves it? In this paper, we analyze how restrictions…
Language Statistics and False Belief Reasoning: Evidence from 41 Open-Weight LMs
Sean Trott, Samuel Taylor, Cameron Jones +2
Research on mental state reasoning in language models (LMs) has the potential to inform theories of human social cognition--such as the theory that mental state reasoning emerges i…
Capacity Constraints and the Multilingual Penalty for Lexical Disambiguation
Sean Trott, Pamela D. Rivière
Multilingual language models (LMs) sometimes under-perform their monolingual counterparts, possibly due to capacity limitations. We quantify this ``multilingual penalty'' for lexic…
Start Making Sense(s): A Developmental Probe of Attention Specialization Using Lexical Ambiguity
Pamela D. Rivière, Sean Trott
Despite an in-principle understanding of self-attention matrix operations in Transformer language models (LMs), it remains unclear precisely how these operations map onto interpret…
Measuring and Modifying the Readability of English Texts with GPT-4
Sean Trott, Pamela D. Rivière
The success of Large Language Models (LLMs) in other domains has raised the question of whether LLMs can reliably assess and manipulate the readability of text. We approach this qu…
Evaluating Contextualized Representations of (Spanish) Ambiguous Words: A New Lexical Resource and Empirical Analysis
Pamela D. Rivière, Anne L. Beatty-Martínez, Sean Trott
Lexical ambiguity -- where a single wordform takes on distinct, context-dependent meanings -- serves as a useful tool to compare across different language models' (LMs') ability to…
Different Tokenization Schemes Lead to Comparable Performance in Spanish Number Agreement
Catherine Arnett, Pamela D. Rivière, Tyler A. Chang +1
The relationship between language model tokenization and performance is an open area of research. Here, we investigate how different tokenization schemes impact number agreement in…