13 papers
Linguistic Context Recodes Visual Representations in Vision-Language Models
Brian Song, Michael A. Lepori, Ellie Pavlick
Goal-directed visual processing is a hallmark of human visual intelligence, resulting in representations that support downstream tasks such as categorization or search. Though visi…
Interpreting Brain Responses to Language with Sparse Features from Language Models
Michael A. Lepori, Kendrick Kay, Greta Tuckute
A central goal of cognitive neuroscience is to characterize the features that are represented by human language cortex. Artificial language models (LMs) have emerged as a powerful…
Into the Rabbit Hull: From Task-Relevant Concepts in DINO to Minkowski Geometry
Thomas Fel, Binxu Wang, Michael A. Lepori +8
DINOv2 is routinely deployed to recognize objects, scenes, and actions; yet the nature of what it perceives remains unknown. As a working baseline, we adopt the Linear Representati…
Language Models Struggle to Use Representations Learned In-Context
Michael A. Lepori, Tal Linzen, Ann Yuan +1
Though large language models (LLMs) have enabled great success across a wide variety of tasks, they still appear to fall short of one of the loftier goals of artificial intelligenc…
Is This Just Fantasy? Language Model Representations Reflect Human Judgments of Event Plausibility
Michael A. Lepori, Jennifer Hu, Ishita Dasgupta +3
Language models (LMs) are used for a diverse range of tasks, from question answering to writing fantastical stories. In order to reliably accomplish these tasks, LMs must be able t…
I Walk the Line: Examining the Role of Gestalt Continuity in Object Binding for Vision Transformers
Alexa R. Tartaglini, Michael A. Lepori
Object binding is a foundational process in visual cognition, during which low-level perceptual features are joined into object representations. Binding has been considered a funda…