50 citations · 63 across the 5 of their papers we have counts for
5 papers · 1 filter
Late Transformer Layers Recode Syntax Canonically: Evidence from Greek Scrambling and Cross-Layer Generalisation
Christos Nikolaos Zacharopoulos, Revekka Kyriakoglou, Chara Tsoukala +1
Probing studies have established that syntactic information is decodable in early and middle transformer layers, but what happens to that information in later layers remains poorly…
Assessing the influence of attractor-verb distance on grammatical agreement in humans and language models
Christos-Nikolaos Zacharopoulos, Théo Desbordes, Mathias Sablé-Meyer
Subject-verb agreement in the presence of an attractor noun located between the main noun and the verb elicits complex behavior: judgments of grammaticality are modulated by the gr…
Causal Transformers Perform Below Chance on Recursive Nested Constructions, Unlike Humans
Yair Lakretz, Théo Desbordes, Dieuwke Hupkes +1
Recursive processing is considered a hallmark of human linguistic abilities. A recent study evaluated recursive processing in recurrent neural language models (RNN-LMs) and showed…
Can RNNs learn Recursive Nested Subject-Verb Agreements?
Yair Lakretz, Théo Desbordes, Jean-Rémi King +3
One of the fundamental principles of contemporary linguistics states that language processing requires the ability to extract recursively nested tree structures. However, it remain…
The emergence of number and syntax units in LSTM language models
Yair Lakretz, German Kruszewski, Theo Desbordes +3
Recent work has shown that LSTMs trained on a generic language modeling objective capture syntax-sensitive generalizations such as long-distance number agreement. We have however n…