565 citations · 707 across the 21 of their papers we have counts for
17 papers · 1 filter
Binding Phoneme Identity to Position in Neural Models and the Human Brain
Nafiseh Momeni, Malo Renaudin, Itzhak Fried +1
A fundamental challenge in cognitive science and neural computation is the binding problem. Compositional generalization can be achieved by factorizing representations into roles,…
OpenWhistle: A Large-Scale Longitudinal Dataset and Benchmark of Bottlenose Dolphin Vocalizations
Faadil Mustun, Chiara Semenzin, Roberto Dessi +7
Recent advances in bioacoustics have been driven by large-scale corpora and standardized benchmarks, yet existing resources are overwhelmingly bird-centric and shallow per species,…
Polar probe linearly decodes semantic structures from LLMs
Pablo J. Diego-Simón, Pierre Orhan, Emmanuel Chemla +2
How do artificial neural networks bind concepts to form complex semantic structures? Here, we propose a simple neural code, whereby the existence and the type of relations between…
Probing Syntax in Large Language Models: Successes and Remaining Challenges
Pablo J. Diego-Simón, Emmanuel Chemla, Jean-Rémi King +1
The syntactic structures of sentences can be readily read-out from the activations of large language models (LLMs). However, the ``structural probes'' that have been developed to r…
A Neural Model for Word Repetition
Daniel Dager, Robin Sobczyk, Emmanuel Chemla +1
It takes several years for the developing brain of a baby to fully master word repetition-the task of hearing a word and repeating it aloud. Repeating a new word, such as from a ne…
Meta-Learning Neural Mechanisms rather than Bayesian Priors
Michael Goodale, Salvador Mascarenhas, Yair Lakretz
Children acquire language despite being exposed to several orders of magnitude less data than large language models require. Meta-learning has been proposed as a way to integrate h…
A polar coordinate system represents syntax in large language models
Pablo Diego-Simón, Stéphane D'Ascoli, Emmanuel Chemla +2
Originally formalized with symbolic representations, syntactic trees may also be effectively represented in the activations of large language models (LLMs). Indeed, a 'Structural P…
What Makes Two Language Models Think Alike?
Louis Jalouzot, Christophe Pallier, Emmanuel Chemla +1
Do architectural and training differences influence the way models represent and process language? Traditional similarity metrics tell us whether two models share a similar represe…
Metric Learning Encoding Models: A Multivariate Framework for Interpreting Neural Representations
Louis Jalouzot, Christophe Pallier, Emmanuel Chemla +1
Understanding how explicit theoretical features are encoded in opaque neural systems is a central challenge now common to neuroscience and AI. We introduce Metric Learning Encoding…
Language acquisition: do children and language models follow similar learning stages?
Linnea Evanson, Yair Lakretz, Jean-Rémi King
During language acquisition, children follow a typical sequence of learning stages, whereby they first learn to categorize phonemes before they develop their lexicon and eventually…
Probing Brain Context-Sensitivity with Masked-Attention Generation
Alexandre Pasquiou, Yair Lakretz, Bertrand Thirion +1
Two fundamental questions in neurolinguistics concerns the brain regions that integrate information beyond the lexical level, and the size of their window of integration. To addres…
Information-Restricted Neural Language Models Reveal Different Brain Regions' Sensitivity to Semantics, Syntax and Context
Alexandre Pasquiou, Yair Lakretz, Bertrand Thirion +1
A fundamental question in neurolinguistics concerns the brain regions involved in syntactic and semantic processing during speech comprehension, both at the lexical (word processin…
Beyond the Imitation Game: Quantifying and extrapolating the capabilities of language models
Aarohi Srivastava, Abhinav Rastogi, Abhishek Rao +448
Language models demonstrate both quantitative improvement and new qualitative capabilities with increasing scale. Despite their potentially transformative impact, these new capabil…
Causal Transformers Perform Below Chance on Recursive Nested Constructions, Unlike Humans
Yair Lakretz, Théo Desbordes, Dieuwke Hupkes +1
Recursive processing is considered a hallmark of human linguistic abilities. A recent study evaluated recursive processing in recurrent neural language models (RNN-LMs) and showed…
Can RNNs learn Recursive Nested Subject-Verb Agreements?
Yair Lakretz, Théo Desbordes, Jean-Rémi King +3
One of the fundamental principles of contemporary linguistics states that language processing requires the ability to extract recursively nested tree structures. However, it remain…
Mechanisms for Handling Nested Dependencies in Neural-Network Language Models and Humans
Yair Lakretz, Dieuwke Hupkes, Alessandra Vergallito +3
Recursive processing in sentence comprehension is considered a hallmark of human linguistic abilities. However, its underlying neural mechanisms remain largely unknown. We studied…
The emergence of number and syntax units in LSTM language models
Yair Lakretz, German Kruszewski, Theo Desbordes +3
Recent work has shown that LSTMs trained on a generic language modeling objective capture syntax-sensitive generalizations such as long-distance number agreement. We have however n…