4 papers
Clozing the Gap: Exploring Why Language Model Surprisal Outperforms Cloze Surprisal
Sathvik Nair, Byung-Doh Oh
How predictable a word is can be quantified in two ways: using human responses to the cloze task or using probabilities from language models (LMs).When used as predictors of proces…
Filling in the Mechanisms: How do LMs Learn Filler-Gap Dependencies under Developmental Constraints?
Atrey Desai, Sathvik Nair
For humans, filler-gap dependencies require a shared representation across different syntactic constructions. Although causal analyses suggest this may also be true for LLMs (Bogur…
Generalizations across filler-gap dependencies in neural language models
Katherine Howitt, Sathvik Nair, Allison Dods +1
Humans develop their grammars by making structural generalizations from finite input. We ask how filler-gap dependencies, which share a structural generalization despite diverse su…
A Psycholinguistic Evaluation of Language Models' Sensitivity to Argument Roles
Eun-Kyoung Rosa Lee, Sathvik Nair, Naomi Feldman
We present a systematic evaluation of large language models' sensitivity to argument roles, i.e., who did what to whom, by replicating psycholinguistic studies on human argument ro…