27 citations · 31 across the 6 of their papers we have counts for
6 papers · 1 filter
A Language Model with Limited Memory Capacity Captures Interference in Human Sentence Processing
William Timkey, Tal Linzen
Two of the central factors believed to underpin human sentence processing difficulty are expectations and retrieval from working memory. A recent attempt to create a unified cognit…
Verb Conjugation in Transformers Is Determined by Linear Encodings of Subject Number
Sophie Hao, Tal Linzen
Deep architectures such as Transformers are sometimes criticized for having uninterpretable "black-box" representations. We use causal intervention analysis to show that, in fact,…
SLOG: A Structural Generalization Benchmark for Semantic Parsing
Bingzhi Li, Lucia Donatelli, Alexander Koller +3
The goal of compositional generalization benchmarks is to evaluate how well models generalize to new complex linguistic expressions. Existing benchmarks often focus on lexical gene…
How to Plant Trees in Language Models: Data and Architectural Effects on the Emergence of Syntactic Inductive Biases
Aaron Mueller, Tal Linzen
Accurate syntactic representations are essential for robust generalization in natural language. Recent work has found that pre-training can teach language models to rely on hierarc…
Improving Compositional Generalization with Latent Structure and Data Augmentation
Linlu Qiu, Peter Shaw, Panupong Pasupat +4
Generic unstructured neural networks have been shown to struggle on out-of-distribution compositional generalization. Compositional data augmentation via example recombination has…
Assessing the Ability of LSTMs to Learn Syntax-Sensitive Dependencies
Tal Linzen, Emmanuel Dupoux, Yoav Goldberg
The success of long short-term memory (LSTM) neural networks in language processing is typically attributed to their ability to capture long-distance statistical regularities. Ling…