7 citations · 13 across the 12 of their papers we have counts for
13 papers
Semantics or spelling? Probing contextual word embeddings with orthographic noise
Jacob A. Matthews, John R. Starr, Marten van Schijndel
Pretrained language model (PLM) hidden states are frequently employed as contextual word embeddings (CWE): high-dimensional representations that encode semantic information given l…
Does Dependency Locality Predict Non-canonical Word Order in Hindi?
Sidharth Ranjan, Marten van Schijndel
Previous work has shown that isolated non-canonical sentences with Object-before-Subject (OSV) order are initially harder to process than their canonical counterparts with Subject-…
Discourse Context Predictability Effects in Hindi Word Order
Sidharth Ranjan, Marten van Schijndel, Sumeet Agarwal +1
We test the hypothesis that discourse predictability influences Hindi syntactic choice. While prior work has shown that a number of factors (e.g., information status, dependency le…
Dual Mechanism Priming Effects in Hindi Word Order
Sidharth Ranjan, Marten van Schijndel, Sumeet Agarwal +1
Word order choices during sentence production can be primed by preceding sentences. In this work, we test the DUAL MECHANISM hypothesis that priming is driven by multiple different…
All Bark and No Bite: Rogue Dimensions in Transformer Language Models Obscure Representational Quality
William Timkey, Marten van Schijndel
Similarity measures are a vital tool for understanding how language models represent and process language. Standard representational similarity measures such as cosine similarity a…
To Point or Not to Point: Understanding How Abstractive Summarizers Paraphrase Text
Matt Wilber, William Timkey, Marten Van Schijndel
Abstractive neural summarization models have seen great improvements in recent years, as shown by ROUGE scores of the generated summaries. But despite these improved metrics, there…
Uncovering Constraint-Based Behavior in Neural Models via Targeted Fine-Tuning
Forrest Davis, Marten van Schijndel
A growing body of literature has focused on detailing the linguistic knowledge embedded in large, pretrained language models. Existing work has shown that non-linguistic biases in…
Discourse structure interacts with reference but not syntax in neural language models
Forrest Davis, Marten van Schijndel
Language models (LMs) trained on large quantities of text have been claimed to acquire abstract linguistic representations. Our work tests the robustness of these abstractions by f…
Recurrent Neural Network Language Models Always Learn English-Like Relative Clause Attachment
Forrest Davis, Marten van Schijndel
A standard approach to evaluating language models analyzes how models assign probabilities to valid versus invalid syntactic constructions (i.e. is a grammatical sentence more prob…
Using Priming to Uncover the Organization of Syntactic Representations in Neural Language Models
Grusha Prasad, Marten van Schijndel, Tal Linzen
Neural language models (LMs) perform well on tasks that require sensitivity to syntactic structure. Drawing on the syntactic priming paradigm from psycholinguistics, we propose a n…
Quantity doesn't buy quality syntax with neural language models
Marten van Schijndel, Aaron Mueller, Tal Linzen
Recurrent neural networks can learn to predict upcoming words remarkably well on average; in syntactically complex contexts, however, they often assign unexpectedly high probabilit…
Can Entropy Explain Successor Surprisal Effects in Reading?
Marten van Schijndel, Tal Linzen
Human reading behavior is sensitive to surprisal: more predictable words tend to be read faster. Unexpectedly, this applies not only to the surprisal of the word that is currently…
A Neural Model of Adaptation in Reading
Marten van Schijndel, Tal Linzen
It has been argued that humans rapidly adapt their lexical and syntactic expectations to match the statistics of the current linguistic context. We provide further support to this…