91 citations · 189 across the 25 of their papers we have counts for
19 papers · 1 filter
Shared circuits predict whether LLMs generalize across formats in arithmetic reasoning
Andrea Gregor de Varda, Sana Pandey, Pengrui Han +2
In many forms of reasoning, including arithmetic reasoning, generalizing across superficial changes in input format is effortless for humans: anyone who can solve 2+5 can also solv…
Different types of syntactic agreement recruit the same units within large language models
Daria Kryvosheieva, Andrea de Varda, Evelina Fedorenko +1
Large language models (LLMs) can reliably distinguish grammatical from ungrammatical sentences, but how grammatical knowledge is represented within the models remains an open quest…
What does it mean to understand language?
Colton Casto, Anna Ivanova, Evelina Fedorenko +1
Language understanding entails not just extracting the surface-level meaning of the linguistic input, but constructing rich mental models of the situation it describes. Here we pro…
Modeling the language cortex with form-independent and enriched representations of sentence meaning reveals remarkable semantic abstractness
Shreya Saha, Shurui Li, Greta Tuckute +5
The human language system represents both linguistic forms and meanings, but the abstractness of the meaning representations remains debated. Here, we searched for abstract represe…
Representing Speech Through Autoregressive Prediction of Cochlear Tokens
Greta Tuckute, Klemen Kotar, Evelina Fedorenko +1
We introduce AuriStream, a biologically inspired model for encoding speech via a two-stage framework inspired by the human auditory processing hierarchy. The first stage transforms…
Language models align with brain regions that represent concepts across modalities
Maria Ryskina, Greta Tuckute, Alexander Fung +2
Cognitive science and neuroscience have long faced the challenge of disentangling representations of language from representations of conceptual meaning. As the same problem arises…
The time scale of redundancy between prosody and linguistic context
Tamar I. Regev, Chiebuka Ohams, Shaylee Xie +5
In spoken communication, information is transmitted not only via words, but also through a rich array of non-verbal signals, including prosody--the non-segmental auditory features…
Elements of World Knowledge (EWoK): A Cognition-Inspired Framework for Evaluating Basic World Knowledge in Language Models
Anna A. Ivanova, Aalok Sathe, Benjamin Lipkin +17
The ability to build and reason about models of the world is essential for situated language understanding. But evaluating world modeling capabilities in modern AI systems -- espec…
Lexicon-Level Contrastive Visual-Grounding Improves Language Modeling
Chengxu Zhuang, Evelina Fedorenko, Jacob Andreas
Today's most accurate language models are trained on orders of magnitude more language data than human language learners receive - but with no supervision from other sensory modali…
Log Probabilities Are a Reliable Estimate of Semantic Plausibility in Base and Instruction-Tuned Language Models
Carina Kauf, Emmanuele Chersoni, Alessandro Lenci +2
Semantic plausibility (e.g. knowing that "the actor won the award" is more likely than "the actor won the battle") serves as an effective proxy for general world knowledge. Languag…
Quantifying the redundancy between prosody and text
Lukas Wolf, Tiago Pimentel, Evelina Fedorenko +4
Prosody -- the suprasegmental component of speech, including pitch, loudness, and tempo -- carries critical aspects of meaning. However, the relationship between the information co…
Large language models implicitly learn to straighten neural sentence trajectories to construct a predictive representation of natural language
Eghbal A. Hosseini, Evelina Fedorenko
Predicting upcoming events is critical to our ability to interact with our environment. Transformer models, trained on next-word prediction, appear to construct representations of…
Visual Grounding Helps Learn Word Meanings in Low-Data Regimes
Chengxu Zhuang, Evelina Fedorenko, Jacob Andreas
Modern neural language models (LMs) are powerful tools for modeling human sentence production and comprehension, and their internal representations are remarkably well-aligned with…
Dissociating language and thought in large language models
Kyle Mahowald, Anna A. Ivanova, Idan A. Blank +3
Large Language Models (LLMs) have come closest among all models to date to mastering human language, yet opinions about their linguistic and cognitive capabilities remain split. He…
A fine-grained comparison of pragmatic language understanding in humans and language models
Jennifer Hu, Sammy Floyd, Olessia Jouravlev +2
Pragmatics and non-literal language understanding are essential to human communication, and present a long-standing challenge for artificial language models. We perform a fine-grai…
Event knowledge in large language models: the gap between the impossible and the unlikely
Carina Kauf, Anna A. Ivanova, Giulia Rambelli +5
Word co-occurrence patterns in language corpora contain a surprising amount of conceptual knowledge. Large language models (LLMs), trained to predict words in context, leverage the…
Grammatical cues to subjecthood are redundant in a majority of simple clauses across languages
Kyle Mahowald, Evgeniia Diachek, Edward Gibson +2
Grammatical cues are sometimes redundant with word meanings in natural language. For instance, English word order rules constrain the word order of a sentence like "The dog chewed…
Semantic projection: recovering human knowledge of multiple, distinct object features from word embeddings
Gabriel Grand, Idan Asher Blank, Francisco Pereira +1
The words of a language reflect the structure of the human mind, allowing us to transmit thoughts between individuals. However, language can represent only a subset of our rich and…
The Natural Stories Corpus
Richard Futrell, Edward Gibson, Hal Tily +4
It is now a common practice to compare models of human language processing by predicting participant reactions (such as reading times) to corpora consisting of rich naturalistic li…