110 citations · 236 across the 25 of their papers we have counts for
28 papers
Testing the Predictions of Surprisal Theory in 11 Languages
Ethan Gotlieb Wilcox, Tiago Pimentel, Clara Meister +2
A fundamental result in psycholinguistics is that less predictable words take a longer time to process. One theoretical explanation for this finding is Surprisal Theory (Hale, 2001…
Probing self-supervised speech models for phonetic and phonemic information: a case study in aspiration
Kinan Martin, Jon Gauthier, Canaan Breiss +1
Textless self-supervised speech models have grown in capabilities in recent years, but the nature of the linguistic information they encode has not yet been thoroughly examined. We…
The neural dynamics of auditory word recognition and integration
Jon Gauthier, Roger Levy
Listeners recognize and integrate words in rapid and noisy everyday speech by combining expectations about upcoming content with incremental sensory evidence. We present a computat…
Prompting is not a substitute for probability measurements in large language models
Jennifer Hu, Roger Levy
Prompting is now a dominant method for evaluating the linguistic knowledge of large language models (LLMs). While other methods directly read out models' probability distributions…
Expectations over Unspoken Alternatives Predict Pragmatic Inferences
Jennifer Hu, Roger Levy, Judith Degen +1
Scalar inferences (SI) are a signature example of how humans interpret language based on unspoken alternatives. While empirical studies have demonstrated that human SI rates are hi…
Language model acceptability judgements are not always robust to context
Koustuv Sinha, Jon Gauthier, Aaron Mueller +4
Targeted syntactic evaluations of language models ask whether models show stable preferences for syntactically acceptable content over minimal-pair unacceptable inputs. Most target…
Probing for Incremental Parse States in Autoregressive Language Models
Tiwalayo Eisape, Vineet Gangireddy, Roger P. Levy +1
Next-word predictions from autoregressive neural language models show remarkable sensitivity to syntax. This work evaluates the extent to which this behavior arises as a result of…
On the Effect of Anticipation on Reading Times
Tiago Pimentel, Clara Meister, Ethan G. Wilcox +2
Over the past two decades, numerous studies have demonstrated how less predictable (i.e., higher surprisal) words take more time to read. In general, these studies have implicitly…
Towards Human-Agent Communication via the Information Bottleneck Principle
Mycal Tucker, Julie Shah, Roger Levy +1
Emergent communication research often focuses on optimizing task-specific utility as a driver for communication. However, human languages appear to evolve under pressure to efficie…
How Adults Understand What Young Children Say
Stephan C. Meylan, Ruthe Foushee, Nicole H. Wong +2
Children's early speech often bears little resemblance to that of adults, and yet parents and other caregivers are able to interpret that speech and react accordingly. Here we inve…
When Does Syntax Mediate Neural Language Model Performance? Evidence from Dropout Probes
Mycal Tucker, Tiwalayo Eisape, Peng Qian +2
Recent causal probing literature reveals when language models and syntactic probes use similar representations. Such techniques may yield "false negative" causality results: models…
Analyzing Wrap-Up Effects through an Information-Theoretic Lens
Clara Meister, Tiago Pimentel, Thomas Hikaru Clark +2
Numerous analyses of reading time (RT) data have been implemented -- all in an effort to better understand the cognitive processes driving reading comprehension. However, data meas…
Grammar-Based Grounded Lexicon Learning
Jiayuan Mao, Haoyue Shi, Jiajun Wu +2
We present Grammar-Based Grounded Lexicon Learning (G2L2), a lexicalist approach toward learning a compositional and grounded meaning representation of language from grounded data,…
Revisiting the Uniform Information Density Hypothesis
Clara Meister, Tiago Pimentel, Patrick Haller +3
The uniform information density (UID) hypothesis posits a preference among language users for utterances structured such that information is distributed uniformly across a signal.…
Controlled Evaluation of Grammatical Knowledge in Mandarin Chinese Language Models
Yiwen Wang, Jennifer Hu, Roger Levy +1
Prior work has shown that structural supervision helps English language models learn generalizations about syntactic phenomena such as subject-verb agreement. However, it remains u…
Structural Guidance for Transformer Language Models
Peng Qian, Tahira Naseem, Roger Levy +1
Transformer-based language models pre-trained on large amounts of text data have proven remarkably successful in learning generic transferable linguistic representations. Here we s…
A Targeted Assessment of Incremental Processing in Neural LanguageModels and Humans
Ethan Gotlieb Wilcox, Pranali Vani, Roger P. Levy
We present a targeted, scaled-up comparison of incremental processing in humans and neural language models by collecting by-word reaction time data for sixteen different syntactic…
What if This Modified That? Syntactic Interventions via Counterfactual Embeddings
Mycal Tucker, Peng Qian, Roger Levy
Neural language models exhibit impressive performance on a variety of tasks, but their internal reasoning may be difficult to understand. Prior art aims to uncover meaningful prope…
Learning Evolved Combinatorial Symbols with a Neuro-symbolic Generative Model
Matthias Hofer, Tuan Anh Le, Roger Levy +1
Humans have the ability to rapidly understand rich combinatorial concepts from limited data. Here we investigate this ability in the context of auditory signals, which have been ev…
Child-directed Listening: How Caregiver Inference Enables Children's Early Verbal Communication
Stephan C. Meylan, Ruthe Foushee, Elika Bergelson +1
How do adults understand children's speech? Children's productions over the course of language development often bear little resemblance to typical adult pronunciations, yet caregi…
Investigating Novel Verb Learning in BERT: Selectional Preference Classes and Alternation-Based Syntactic Generalization
Tristan Thrush, Ethan Wilcox, Roger Levy
Previous studies investigating the syntactic abilities of deep learning models have not targeted the relationship between the strength of the grammatical generalization and the amo…
Bridging Information-Seeking Human Gaze and Machine Reading Comprehension
Jonathan Malmaud, Roger Levy, Yevgeni Berzak
In this work, we analyze how human gaze during reading comprehension is conditioned on the given reading comprehension question, and whether this signal can be beneficial for machi…
Structural Supervision Improves Few-Shot Learning and Syntactic Generalization in Neural Language Models
Ethan Wilcox, Peng Qian, Richard Futrell +3
Humans can learn structural properties about a word from minimal experience, and deploy their learned syntactic representations uniformly in different grammatical contexts. We asse…
On the Predictive Power of Neural Language Models for Human Real-Time Comprehension Behavior
Ethan Gotlieb Wilcox, Jon Gauthier, Jennifer Hu +2
Human reading behavior is tuned to the statistics of natural language: the time it takes human subjects to read a word can be predicted from estimates of the word's probability in…