5 papers
How Open Must Language Models be to Enable Reliable Scientific Inference?
James A. Michaelov, Catherine Arnett, Tyler A. Chang +7
How does the extent to which a model is open or closed impact the scientific inferences that can be drawn from research that involves it? In this paper, we analyze how restrictions…
Readers make targeted regressions to plausible errors in reanalysis of "noisy-channel garden-path" sentences
Thomas Hikaru Clark, Roger Levy, Edward Gibson
A key question in psycholinguistics is how inferences about the meaning of linguistic input unfold incrementally a comprehender's mind. In this work, we study reading dynamics for…
Algorithmic Consequences of Particle Filters for Sentence Processing: Amplified Garden-Paths and Digging-In Effects
Amani Maina-Kilaas, Roger Levy
Under surprisal theory, linguistic representations affect processing difficulty only through the bottleneck of surprisal. Our best estimates of surprisal come from large language m…
Revisiting Real-Time Digging-In Effects: No Evidence from NP/Z Garden-Paths
Amani Maina-Kilaas, Roger Levy
Digging-in effects, where disambiguation difficulty increases with longer ambiguous regions, have been cited as evidence for self-organized sentence processing, in which structural…
N-gram-like Language Models Predict Naturalistic Reading Time Best
James A. Michaelov, Roger P. Levy
Recent work has found that contemporary language models such as transformers can become so good at next-word prediction that the probabilities they calculate become worse for predi…