565 citations · 675 across the 33 of their papers we have counts for
39 papers
Analyzing Repetition and Interference Effects in Memory-Constrained User Simulators
Cara Su-Yi Leong, Nicholas Tomlin, Tal Linzen
User simulators can help evaluate and train AI systems to interact with humans at scale. To do so, evaluation of user simulators must measure behavioral parity with people. We eval…
Do Syntax Heads Influence Model Behavior?
William Timkey, Tal Linzen
Transformer language models develop attention heads that specialize in syntactic relations, for example, attending from verbs to their subjects. Do these heads play a causal role i…
Bayesian Fine-tuning Yields Language Models that are as Bayesian as their Beliefs Allow
Polina Tsvilodub, Andreas Waldis, Linlu Qiu +2
Language models (LMs) are increasingly used for tasks that require reasoning about hidden variables from a few observations, for which Bayesian inference is the normatively correct…
CogGym: Towards Large-Scale Comparative Evaluation of Human and Machine Cognition
Lance Ying, Jinzhou Wu, Yingshan Susan Wang +53
Understanding and modeling human intelligence are parallel goals shared by artificial intelligence (AI) and cognitive science. As AI systems grow increasingly capable, in what ways…
Does Anthropomorphic Language Impact Public Perceptions of AI?
Betty Li Hou, Sophie Hao, Sunoo Park +1
Public discourse about artificial intelligence (AI) often uses anthropomorphic language: language that attributes human capabilities and characteristics to AI systems. This practic…
Simulating Human Memory with Language Models
Qihan Wang, Nicholas Tomlin, Michael Hu +2
Language models are increasingly being deployed as user simulators, but their memory is far more reliable than that of real users. To measure this gap, we run a series of classic m…
Why are language models less surprised than humans? Testing the Parse Multiplicity Mismatch Hypothesis
William Timkey, Brian Dillon, Tal Linzen
Surprisal theory posits that the processing difficulty of a word is determined by its predictability in context, offering a potential link between human sentence processing and nex…
Always Learning, Always Mixing: Efficient and Simple Data Mixing All The Time
Michael Y. Hu, Apurva Gandhi, Kyunghyun Cho +2
Data mixing decides how to combine different sources or types of data and is a consequential problem throughout language model training. In pretraining, data composition is a key d…
Can LLMs Introspect? A Reality Check
Shashwat Singh, Tal Linzen, Shauli Ravfogel
Can large language models detect and report their own internal states? A number of recent studies have argued that they can. Drawing on lessons from human metacognition research, w…
Evaluating In-Context Translation with Synchronous Context-Free Grammar Transduction
Jackson Petty, Jaulie Goe, Tal Linzen
Low-resource languages pose a challenge for machine translation with large language models (LLMs), which require large amounts of training data. One potential way to circumvent thi…
BabyLM Turns 4 and Goes Multilingual: Call for Papers for the 2026 BabyLM Workshop
Leshem Choshen, Ryan Cotterell, Mustafa Omer Gul +7
The goal of the BabyLM is to stimulate new research connections between cognitive modeling and language model pretraining. We invite contributions in this vein to the BabyLM Worksh…
LLM surprisal is necessary but not sufficient to capture English garden-path effects: Evidence from joint latent modeling of reading paradigms
Dario Paape, Tal Linzen, Shravan Vasishth
Temporarily ambiguous garden-path sentences ("While the team trained the striker wondered... ") are known to cause processing difficulty, which can manifest itself in a variety of…
Language Models Struggle to Use Representations Learned In-Context
Michael A. Lepori, Tal Linzen, Ann Yuan +1
Though large language models (LLMs) have enabled great success across a wide variety of tasks, they still appear to fall short of one of the loftier goals of artificial intelligenc…
Emergence of Linear Truth Encodings in Language Models
Shauli Ravfogel, Gilad Yehudai, Tal Linzen +2
Recent probing studies reveal that large language models exhibit linear subspaces that separate true from false statements, yet the mechanism behind their emergence is unclear. We…
To model human linguistic prediction, make LLMs less superhuman
Byung-Doh Oh, Tal Linzen
When we read, we make predictions about upcoming words; these predictions influence our reading behavior. The success of large language models (LLMs), which, like humans, make pred…
Beyond the Rosetta Stone: Unification Forces in Generalization Dynamics
Carter Blum, Katja Filippova, Ann Yuan +8
Large language models (LLMs) struggle with cross-lingual knowledge transfer: they sometimes hallucinate when asked in one language about facts expressed in a different language dur…
RELIC: Evaluating Complex Reasoning via the Recognition of Languages In-Context
Jackson Petty, Michael Y. Hu, Wentao Wang +3
Large language models (LLMs) are increasingly used to solve complex tasks where they must retrieve and compose many pieces of in-context information in long reasoning chains. For m…
Multilingual Prompting for Improving LLM Generation Diversity
Qihan Wang, Shidong Pan, Tal Linzen +1
Large Language Models (LLMs) are known to lack cultural representation and overall diversity in their generations, from expressing opinions to answering factual questions. To mitig…
Findings of the BabyLM Challenge: Sample-Efficient Pretraining on Developmentally Plausible Corpora
Alex Warstadt, Aaron Mueller, Leshem Choshen +8
Children can acquire language from less than 100 million words of input. Large language models are far less data-efficient: they typically require 3 or 4 orders of magnitude more d…
Bayesian Teaching Enables Probabilistic Reasoning in Large Language Models
Linlu Qiu, Fei Sha, Kelsey Allen +3
Large language models (LLMs) are increasingly used as agents that interact with users and with the world. To do so successfully, LLMs must construct representations of the world an…
BabyLM Turns 3: Call for papers for the 2025 BabyLM workshop
Lucas Charpentier, Leshem Choshen, Ryan Cotterell +11
BabyLM aims to dissolve the boundaries between cognitive modeling and language modeling. We call for both workshop papers and for researchers to join the 3rd BabyLM competition. As…
Between Circuits and Chomsky: Pre-pretraining on Formal Languages Imparts Linguistic Biases
Michael Y. Hu, Jackson Petty, Chuan Shi +2
Pretraining language models on formal language can improve their acquisition of natural language. Which features of the formal language impart an inductive bias that leads to effec…
Rapid Word Learning Through Meta In-Context Learning
Wentao Wang, Guangyuan Jiang, Tal Linzen +1
Humans can quickly learn a new word from a few illustrative examples, and then systematically and flexibly use it in novel contexts. Yet the abilities of current language models fo…
Findings of the Second BabyLM Challenge: Sample-Efficient Pretraining on Developmentally Plausible Corpora
Michael Y. Hu, Aaron Mueller, Candace Ross +7
The BabyLM Challenge is a community effort to close the data-efficiency gap between human and computational language learners. Participants compete to optimize language model train…