110 citations · 201 across the 9 of their papers we have counts for
15 papers
When Does Syntax Mediate Neural Language Model Performance? Evidence from Dropout Probes
Mycal Tucker, Tiwalayo Eisape, Peng Qian +2
Recent causal probing literature reveals when language models and syntactic probes use similar representations. Such techniques may yield "false negative" causality results: models…
Revisiting the Uniform Information Density Hypothesis
Clara Meister, Tiago Pimentel, Patrick Haller +3
The uniform information density (UID) hypothesis posits a preference among language users for utterances structured such that information is distributed uniformly across a signal.…
Controlled Evaluation of Grammatical Knowledge in Mandarin Chinese Language Models
Yiwen Wang, Jennifer Hu, Roger Levy +1
Prior work has shown that structural supervision helps English language models learn generalizations about syntactic phenomena such as subject-verb agreement. However, it remains u…
Structural Guidance for Transformer Language Models
Peng Qian, Tahira Naseem, Roger Levy +1
Transformer-based language models pre-trained on large amounts of text data have proven remarkably successful in learning generic transferable linguistic representations. Here we s…
What if This Modified That? Syntactic Interventions via Counterfactual Embeddings
Mycal Tucker, Peng Qian, Roger Levy
Neural language models exhibit impressive performance on a variety of tasks, but their internal reasoning may be difficult to understand. Prior art aims to uncover meaningful prope…
Learning Evolved Combinatorial Symbols with a Neuro-symbolic Generative Model
Matthias Hofer, Tuan Anh Le, Roger Levy +1
Humans have the ability to rapidly understand rich combinatorial concepts from limited data. Here we investigate this ability in the context of auditory signals, which have been ev…