125 citations · 584 across the 45 of their papers we have counts for
51 papers
The Emergent Symbolic Structure of Artificial Neural Networks
R. Thomas McCoy, Paul Soulos, Tal Linzen +1
Modern systems in artificial intelligence (AI) somehow excel in domains for which they seem poorly suited. Intelligence has traditionally been modeled as operating over structured…
Manipulating language models' training data to study syntactic constraint learning: the case of English passivization
Cara Su-Yi Leong, Tal Linzen
Grammatical rules in natural languages are often characterized by exceptions. How do language learners learn these exceptions to otherwise general patterns? Here, we study this que…
SPAWNing Structural Priming Predictions from a Cognitively Motivated Parser
Grusha Prasad, Tal Linzen
Structural priming is a widely used psycholinguistic paradigm to study human sentence representations. In this work we introduce SPAWN, a cognitively motivated parser that can gene…
Do Language Models' Words Refer?
Matthew Mandelkern, Tal Linzen
What do language models (LMs) do with language? Everyone agrees that they can produce sequences of (mostly) coherent strings of English. But do those sentences mean something, or a…
Language Models Can Learn Exceptions to Syntactic Rules
Cara Su-Yi Leong, Tal Linzen
Artificial neural networks can generalize productively to novel contexts. Can they also learn exceptions to those productive rules? We explore this question using the case of restr…
How to Plant Trees in Language Models: Data and Architectural Effects on the Emergence of Syntactic Inductive Biases
Aaron Mueller, Tal Linzen
Accurate syntactic representations are essential for robust generalization in natural language. Recent work has found that pre-training can teach language models to rely on hierarc…
How poor is the stimulus? Evaluating hierarchical generalization in neural networks trained on child-directed speech
Aditya Yedetore, Tal Linzen, Robert Frank +1
When acquiring syntax, children consistently choose hierarchical rules over competing non-hierarchical possibilities. Is this preference due to a learning bias for hierarchical str…
Uncontrolled Lexical Exposure Leads to Overestimation of Compositional Generalization in Pretrained Models
Najoung Kim, Tal Linzen, Paul Smolensky
Human linguistic capacity is often characterized by compositionality and the generalization it enables -- human learners can produce and comprehend novel complex expressions by com…
Causal Analysis of Syntactic Agreement Neurons in Multilingual Language Models
Aaron Mueller, Yu Xia, Tal Linzen
Structural probing work has found evidence for latent syntactic information in pre-trained language models. However, much of this analysis has focused on monolingual models, and an…
Characterizing Verbatim Short-Term Memory in Neural Language Models
Kristijan Armeni, Christopher Honey, Tal Linzen
When a language model is trained to predict natural language sequences, its prediction at each moment depends on a representation of prior context. What kind of information about t…
Syntactic Surprisal From Neural Models Predicts, But Underestimates, Human Processing Difficulty From Syntactic Ambiguities
Suhas Arehalli, Brian Dillon, Tal Linzen
Humans exhibit garden path effects: When reading sentences that are temporarily structurally ambiguous, they slow down when the structure is disambiguated in favor of the less pref…
Entailment Semantics Can Be Extracted from an Ideal Language Model
William Merrill, Alex Warstadt, Tal Linzen
Language models are often trained on text alone, without additional grounding. There is debate as to how much of natural language semantics can be inferred from such a procedure. W…
When a sentence does not introduce a discourse entity, Transformer-based models still sometimes refer to it
Sebastian Schuster, Tal Linzen
Understanding longer narratives or participating in conversations requires tracking of discourse entities that have been mentioned. Indefinite noun phrases (NPs), such as 'a dog',…
Coloring the Blank Slate: Pre-training Imparts a Hierarchical Inductive Bias to Sequence-to-sequence Models
Aaron Mueller, Robert Frank, Tal Linzen +2
Relations between words are governed by hierarchical structure rather than linear ordering. Sequence-to-sequence (seq2seq) models, despite their success in downstream NLP applicati…
Improving Compositional Generalization with Latent Structure and Data Augmentation
Linlu Qiu, Peter Shaw, Panupong Pasupat +4
Generic unstructured neural networks have been shown to struggle on out-of-distribution compositional generalization. Compositional data augmentation via example recombination has…
Frequency Effects on Syntactic Rule Learning in Transformers
Jason Wei, Dan Garrette, Tal Linzen +1
Pre-trained language models perform well on a variety of linguistic tasks that require symbolic reasoning, raising the question of whether such models implicitly represent abstract…
NOPE: A Corpus of Naturally-Occurring Presuppositions in English
Alicia Parrish, Sebastian Schuster, Alex Warstadt +5
Understanding language requires grasping not only the overtly stated content, but also making inferences about things that were left unsaid. These inferences include presupposition…
Causal Analysis of Syntactic Agreement Mechanisms in Neural Language Models
Matthew Finlayson, Aaron Mueller, Sebastian Gehrmann +3
Targeted syntactic evaluations have demonstrated the ability of language models to perform subject-verb agreement given difficult contexts. To elucidate the mechanisms by which the…
Counterfactual Interventions Reveal the Causal Effect of Relative Clause Representations on Agreement Prediction
Shauli Ravfogel, Grusha Prasad, Tal Linzen +1
When language models process syntactically complex sentences, do they use their representations of syntax in a manner that is consistent with the grammar of the language? We propos…
Evaluating Attribution in Dialogue Systems: The BEGIN Benchmark
Nouha Dziri, Hannah Rashkin, Tal Linzen +1
Knowledge-grounded dialogue systems powered by large language models often generate responses that, while fluent, are not attributable to a relevant source of information. Progress…
Does Putting a Linguist in the Loop Improve NLU Data Collection?
Alicia Parrish, William Huang, Omar Agha +7
Many crowdsourced NLP datasets contain systematic gaps and biases that are identified only after data collection is complete. Identifying these issues from early data samples durin…
COGS: A Compositional Generalization Challenge Based on Semantic Interpretation
Najoung Kim, Tal Linzen
Natural language is characterized by compositionality: the meaning of a complex expression is constructed from the meanings of its constituent parts. To facilitate the evaluation o…
Universal linguistic inductive biases via meta-learning
R. Thomas McCoy, Erin Grant, Paul Smolensky +2
How do learners acquire languages from the limited data available to them? This process must involve some inductive biases - factors that affect how a learner generalizes - but it…
Cross-Linguistic Syntactic Evaluation of Word Prediction Models
Aaron Mueller, Garrett Nicolai, Panayiota Petrou-Zeniou +2
A range of studies have concluded that neural word prediction models can distinguish grammatical from ungrammatical sentences with high accuracy. However, these studies are based p…