110 citations · 203 across the 10 of their papers we have counts for
16 papers · 1 filter
Probing for Incremental Parse States in Autoregressive Language Models
Tiwalayo Eisape, Vineet Gangireddy, Roger P. Levy +1
Next-word predictions from autoregressive neural language models show remarkable sensitivity to syntax. This work evaluates the extent to which this behavior arises as a result of…
When Does Syntax Mediate Neural Language Model Performance? Evidence from Dropout Probes
Mycal Tucker, Tiwalayo Eisape, Peng Qian +2
Recent causal probing literature reveals when language models and syntactic probes use similar representations. Such techniques may yield "false negative" causality results: models…
Revisiting the Uniform Information Density Hypothesis
Clara Meister, Tiago Pimentel, Patrick Haller +3
The uniform information density (UID) hypothesis posits a preference among language users for utterances structured such that information is distributed uniformly across a signal.…
Controlled Evaluation of Grammatical Knowledge in Mandarin Chinese Language Models
Yiwen Wang, Jennifer Hu, Roger Levy +1
Prior work has shown that structural supervision helps English language models learn generalizations about syntactic phenomena such as subject-verb agreement. However, it remains u…
Structural Guidance for Transformer Language Models
Peng Qian, Tahira Naseem, Roger Levy +1
Transformer-based language models pre-trained on large amounts of text data have proven remarkably successful in learning generic transferable linguistic representations. Here we s…
What if This Modified That? Syntactic Interventions via Counterfactual Embeddings
Mycal Tucker, Peng Qian, Roger Levy
Neural language models exhibit impressive performance on a variety of tasks, but their internal reasoning may be difficult to understand. Prior art aims to uncover meaningful prope…