565 citations · 707 across the 21 of their papers we have counts for
24 papers
Binding Phoneme Identity to Position in Neural Models and the Human Brain
Nafiseh Momeni, Malo Renaudin, Itzhak Fried +1
A fundamental challenge in cognitive science and neural computation is the binding problem. Compositional generalization can be achieved by factorizing representations into roles,…
OpenWhistle: A Large-Scale Longitudinal Dataset and Benchmark of Bottlenose Dolphin Vocalizations
Faadil Mustun, Chiara Semenzin, Roberto Dessi +7
Recent advances in bioacoustics have been driven by large-scale corpora and standardized benchmarks, yet existing resources are overwhelmingly bird-centric and shallow per species,…
Dolph2Vec: Self-Supervised Representations of Dolphin Vocalizations
Chiara Semenzin, Faadil Mustun, Roberto Dessi +5
Self-supervised learning (SSL) has opened new opportunities in bioacoustics by enabling scalable modeling of animal vocalizations without the need for expensive manual annotation.…
Polar probe linearly decodes semantic structures from LLMs
Pablo J. Diego-Simón, Pierre Orhan, Emmanuel Chemla +2
How do artificial neural networks bind concepts to form complex semantic structures? Here, we propose a simple neural code, whereby the existence and the type of relations between…
Emergence of Phonemic, Syntactic, and Semantic Representations in Artificial Neural Networks
Pierre Orhan, Pablo Diego-Simón, Emmnanuel Chemla +3
During language acquisition, children successively learn to categorize phonemes, identify words, and combine them with syntax to form new meaning. While the development of this beh…
MapFormer: Self-Supervised Learning of Cognitive Maps with Input-Dependent Positional Embeddings
Victor Rambaud, Salvador Mascarenhas, Yair Lakretz
A cognitive map is an internal model which encodes the abstract relationships among entities in the world, giving humans and animals the flexibility to adapt to new situations, wit…
Probing Syntax in Large Language Models: Successes and Remaining Challenges
Pablo J. Diego-Simón, Emmanuel Chemla, Jean-Rémi King +1
The syntactic structures of sentences can be readily read-out from the activations of large language models (LLMs). However, the ``structural probes'' that have been developed to r…
A Neural Model for Word Repetition
Daniel Dager, Robin Sobczyk, Emmanuel Chemla +1
It takes several years for the developing brain of a baby to fully master word repetition-the task of hearing a word and repeating it aloud. Repeating a new word, such as from a ne…
Optimizing fMRI Data Acquisition for Decoding Natural Speech with Limited Participants
Louis Jalouzot, Alexis Thual, Yair Lakretz +2
We investigate optimal strategies for decoding perceived natural speech from fMRI data acquired from a limited number of participants. Leveraging Lebel et al. (2023)'s dataset of 8…
Meta-Learning Neural Mechanisms rather than Bayesian Priors
Michael Goodale, Salvador Mascarenhas, Yair Lakretz
Children acquire language despite being exposed to several orders of magnitude less data than large language models require. Meta-learning has been proposed as a way to integrate h…
Disentanglement and Compositionality of Letter Identity and Letter Position in Variational Auto-Encoder Vision Models
Bruno Bianchi, Aakash Agrawal, Stanislas Dehaene +2
Human readers can accurately count how many letters are in a word (e.g., 7 in ``buffalo''), remove a letter from a given position (e.g., ``bufflo'') or add a new one. The human bra…
A polar coordinate system represents syntax in large language models
Pablo Diego-Simón, Stéphane D'Ascoli, Emmanuel Chemla +2
Originally formalized with symbolic representations, syntactic trees may also be effectively represented in the activations of large language models (LLMs). Indeed, a 'Structural P…
What Makes Two Language Models Think Alike?
Louis Jalouzot, Christophe Pallier, Emmanuel Chemla +1
Do architectural and training differences influence the way models represent and process language? Traditional similarity metrics tell us whether two models share a similar represe…
Metric Learning Encoding Models: A Multivariate Framework for Interpreting Neural Representations
Louis Jalouzot, Christophe Pallier, Emmanuel Chemla +1
Understanding how explicit theoretical features are encoded in opaque neural systems is a central challenge now common to neuroscience and AI. We introduce Metric Learning Encoding…
Language acquisition: do children and language models follow similar learning stages?
Linnea Evanson, Yair Lakretz, Jean-Rémi King
During language acquisition, children follow a typical sequence of learning stages, whereby they first learn to categorize phonemes before they develop their lexicon and eventually…
Probing Brain Context-Sensitivity with Masked-Attention Generation
Alexandre Pasquiou, Yair Lakretz, Bertrand Thirion +1
Two fundamental questions in neurolinguistics concerns the brain regions that integrate information beyond the lexical level, and the size of their window of integration. To addres…
Information-Restricted Neural Language Models Reveal Different Brain Regions' Sensitivity to Semantics, Syntax and Context
Alexandre Pasquiou, Yair Lakretz, Bertrand Thirion +1
A fundamental question in neurolinguistics concerns the brain regions involved in syntactic and semantic processing during speech comprehension, both at the lexical (word processin…
Neural Language Models are not Born Equal to Fit Brain Data, but Training Helps
Alexandre Pasquiou, Yair Lakretz, John Hale +2
Neural Language Models (NLMs) have made tremendous advances during the last years, achieving impressive performance on various linguistic tasks. Capitalizing on this, studies in ne…
Beyond the Imitation Game: Quantifying and extrapolating the capabilities of language models
Aarohi Srivastava, Abhinav Rastogi, Abhishek Rao +448
Language models demonstrate both quantitative improvement and new qualitative capabilities with increasing scale. Despite their potentially transformative impact, these new capabil…
Causal Transformers Perform Below Chance on Recursive Nested Constructions, Unlike Humans
Yair Lakretz, Théo Desbordes, Dieuwke Hupkes +1
Recursive processing is considered a hallmark of human linguistic abilities. A recent study evaluated recursive processing in recurrent neural language models (RNN-LMs) and showed…
Can RNNs learn Recursive Nested Subject-Verb Agreements?
Yair Lakretz, Théo Desbordes, Jean-Rémi King +3
One of the fundamental principles of contemporary linguistics states that language processing requires the ability to extract recursively nested tree structures. However, it remain…
Mechanisms for Handling Nested Dependencies in Neural-Network Language Models and Humans
Yair Lakretz, Dieuwke Hupkes, Alessandra Vergallito +3
Recursive processing in sentence comprehension is considered a hallmark of human linguistic abilities. However, its underlying neural mechanisms remain largely unknown. We studied…
The emergence of number and syntax units in LSTM language models
Yair Lakretz, German Kruszewski, Theo Desbordes +3
Recent work has shown that LSTMs trained on a generic language modeling objective capture syntax-sensitive generalizations such as long-distance number agreement. We have however n…
Metric Learning for Phoneme Perception
Yair Lakretz, Gal Chechik, Evan-Gary Cohen +2
Metric functions for phoneme perception capture the similarity structure among phonemes in a given language and therefore play a central role in phonology and psycho-linguistics. V…