125 citations · 584 across the 45 of their papers we have counts for
7 papers · 1 filter
Uncontrolled Lexical Exposure Leads to Overestimation of Compositional Generalization in Pretrained Models
Najoung Kim, Tal Linzen, Paul Smolensky
Human linguistic capacity is often characterized by compositionality and the generalization it enables -- human learners can produce and comprehend novel complex expressions by com…
Causal Analysis of Syntactic Agreement Neurons in Multilingual Language Models
Aaron Mueller, Yu Xia, Tal Linzen
Structural probing work has found evidence for latent syntactic information in pre-trained language models. However, much of this analysis has focused on monolingual models, and an…
Characterizing Verbatim Short-Term Memory in Neural Language Models
Kristijan Armeni, Christopher Honey, Tal Linzen
When a language model is trained to predict natural language sequences, its prediction at each moment depends on a representation of prior context. What kind of information about t…
Syntactic Surprisal From Neural Models Predicts, But Underestimates, Human Processing Difficulty From Syntactic Ambiguities
Suhas Arehalli, Brian Dillon, Tal Linzen
Humans exhibit garden path effects: When reading sentences that are temporarily structurally ambiguous, they slow down when the structure is disambiguated in favor of the less pref…
Entailment Semantics Can Be Extracted from an Ideal Language Model
William Merrill, Alex Warstadt, Tal Linzen
Language models are often trained on text alone, without additional grounding. There is debate as to how much of natural language semantics can be inferred from such a procedure. W…
When a sentence does not introduce a discourse entity, Transformer-based models still sometimes refer to it
Sebastian Schuster, Tal Linzen
Understanding longer narratives or participating in conversations requires tracking of discourse entities that have been mentioned. Indefinite noun phrases (NPs), such as 'a dog',…
Coloring the Blank Slate: Pre-training Imparts a Hierarchical Inductive Bias to Sequence-to-sequence Models
Aaron Mueller, Robert Frank, Tal Linzen +2
Relations between words are governed by hierarchical structure rather than linear ordering. Sequence-to-sequence (seq2seq) models, despite their success in downstream NLP applicati…