565 citations · 675 across the 33 of their papers we have counts for
6 papers · 1 filter
In-context Learning Generalizes, But Not Always Robustly: The Case of Syntax
Aaron Mueller, Albert Webson, Jackson Petty +1
In-context learning (ICL) is now a common method for teaching large language models (LLMs) new tasks: given labeled examples in the input context, the LLM learns to perform the tas…
A Systematic Comparison of Syllogistic Reasoning in Humans and Language Models
Tiwalayo Eisape, MH Tessler, Ishita Dasgupta +3
A central component of rational behavior is logical inference: the process of determining which conclusions follow from a set of premises. Psychologists have documented several way…
A Language Model with Limited Memory Capacity Captures Interference in Human Sentence Processing
William Timkey, Tal Linzen
Two of the central factors believed to underpin human sentence processing difficulty are expectations and retrieval from working memory. A recent attempt to create a unified cognit…
Verb Conjugation in Transformers Is Determined by Linear Encodings of Subject Number
Sophie Hao, Tal Linzen
Deep architectures such as Transformers are sometimes criticized for having uninterpretable "black-box" representations. We use causal intervention analysis to show that, in fact,…
SLOG: A Structural Generalization Benchmark for Semantic Parsing
Bingzhi Li, Lucia Donatelli, Alexander Koller +3
The goal of compositional generalization benchmarks is to evaluate how well models generalize to new complex linguistic expressions. Existing benchmarks often focus on lexical gene…
The Impact of Depth on Compositional Generalization in Transformer Language Models
Jackson Petty, Sjoerd van Steenkiste, Ishita Dasgupta +3
To process novel sentences, language models (LMs) must generalize compositionally -- combine familiar elements in new ways. What aspects of a model's structure promote compositiona…