565 citations · 628 across the 7 of their papers we have counts for
7 papers
Late Transformer Layers Recode Syntax Canonically: Evidence from Greek Scrambling and Cross-Layer Generalisation
Christos Nikolaos Zacharopoulos, Revekka Kyriakoglou, Chara Tsoukala +1
Probing studies have established that syntactic information is decodable in early and middle transformer layers, but what happens to that information in later layers remains poorly…
NeuralSet: A High-Performing Python Package for Neuro-AI
Jean-Rémi King, Corentin Bel, Linnea Evanson +25
Artificial intelligence (AI) is increasingly central to understanding how the brain processes information. However, the integration of neuroscience and modern AI is bottlenecked by…
Assessing the influence of attractor-verb distance on grammatical agreement in humans and language models
Christos-Nikolaos Zacharopoulos, Théo Desbordes, Mathias Sablé-Meyer
Subject-verb agreement in the presence of an attractor noun located between the main noun and the verb elicits complex behavior: judgments of grammaticality are modulated by the gr…
Beyond the Imitation Game: Quantifying and extrapolating the capabilities of language models
Aarohi Srivastava, Abhinav Rastogi, Abhishek Rao +448
Language models demonstrate both quantitative improvement and new qualitative capabilities with increasing scale. Despite their potentially transformative impact, these new capabil…
Causal Transformers Perform Below Chance on Recursive Nested Constructions, Unlike Humans
Yair Lakretz, Théo Desbordes, Dieuwke Hupkes +1
Recursive processing is considered a hallmark of human linguistic abilities. A recent study evaluated recursive processing in recurrent neural language models (RNN-LMs) and showed…
Can RNNs learn Recursive Nested Subject-Verb Agreements?
Yair Lakretz, Théo Desbordes, Jean-Rémi King +3
One of the fundamental principles of contemporary linguistics states that language processing requires the ability to extract recursively nested tree structures. However, it remain…
The emergence of number and syntax units in LSTM language models
Yair Lakretz, German Kruszewski, Theo Desbordes +3
Recent work has shown that LSTMs trained on a generic language modeling objective capture syntax-sensitive generalizations such as long-distance number agreement. We have however n…