6 citations · 20 across the 21 of their papers we have counts for
23 papers
Linguistic Context Recodes Visual Representations in Vision-Language Models
Brian Song, Michael A. Lepori, Ellie Pavlick
Goal-directed visual processing is a hallmark of human visual intelligence, resulting in representations that support downstream tasks such as categorization or search. Though visi…
Interpreting Brain Responses to Language with Sparse Features from Language Models
Michael A. Lepori, Kendrick Kay, Greta Tuckute
A central goal of cognitive neuroscience is to characterize the features that are represented by human language cortex. Artificial language models (LMs) have emerged as a powerful…
I Walk the Line: Examining the Role of Gestalt Continuity in Object Binding for Vision Transformers
Alexa R. Tartaglini, Michael A. Lepori
Object binding is a foundational process in visual cognition, during which low-level perceptual features are joined into object representations. Binding has been considered a funda…
Language Models Struggle to Use Representations Learned In-Context
Michael A. Lepori, Tal Linzen, Ann Yuan +1
Though large language models (LLMs) have enabled great success across a wide variety of tasks, they still appear to fall short of one of the loftier goals of artificial intelligenc…
Into the Rabbit Hull: From Task-Relevant Concepts in DINO to Minkowski Geometry
Thomas Fel, Binxu Wang, Michael A. Lepori +8
DINOv2 is routinely deployed to recognize objects, scenes, and actions; yet the nature of what it perceives remains unknown. As a working baseline, we adopt the Linear Representati…
Is This Just Fantasy? Language Model Representations Reflect Human Judgments of Event Plausibility
Michael A. Lepori, Jennifer Hu, Ishita Dasgupta +3
Language models (LMs) are used for a diverse range of tasks, from question answering to writing fantastical stories. In order to reliably accomplish these tasks, LMs must be able t…
Signatures of human-like processing in Transformer forward passes
Jennifer Hu, Michael A. Lepori, Michael Franke
Modern AI models are increasingly being used as theoretical tools to study human cognition. One dominant approach is to evaluate whether human-derived measures are predicted by a m…
Pixels Versus Priors: Controlling Knowledge Priors in Vision-Language Models through Visual Counterfacts
Michal Golovanevsky, William Rudman, Michael Lepori +3
Multimodal Large Language Models (MLLMs) perform well on tasks such as visual question answering, but it remains unclear whether their reasoning relies more on memorized world know…
Are LLMs Models of Distributional Semantics? A Case Study on Quantifiers
Zhang Enyan, Zewei Wang, Michael A. Lepori +2
Distributional semantics is the linguistic theory that a word's meaning can be derived from its distribution in natural language (i.e., its use). Language models are commonly viewe…
Racing Thoughts: Explaining Contextualization Errors in Large Language Models
Michael A. Lepori, Michael C. Mozer, Asma Ghandeharioun
The profound success of transformer-based language models can largely be attributed to their ability to integrate relevant contextual information from an input sequence in order to…
Beyond the Doors of Perception: Vision Transformers Represent Relations Between Objects
Michael A. Lepori, Alexa R. Tartaglini, Wai Keen Vong +3
Though vision transformers (ViTs) have achieved state-of-the-art performance in a variety of settings, they exhibit surprising failures when performing tasks involving visual relat…
Who's asking? User personas and the mechanics of latent misalignment
Asma Ghandeharioun, Ann Yuan, Marius Guerard +3
Despite investments in improving model safety, studies show that misaligned capabilities remain latent in safety-tuned models. In this work, we shed light on the mechanics of this…
Dual Process Learning: Controlling Use of In-Context vs. In-Weights Strategies with Weight Forgetting
Suraj Anand, Michael A. Lepori, Jack Merullo +1
Language models have the ability to perform in-context learning (ICL), allowing them to flexibly adapt their behavior based on context. This contrasts with in-weights learning (IWL…
Uncovering Intermediate Variables in Transformers using Circuit Probing
Michael A. Lepori, Thomas Serre, Ellie Pavlick
Neural network models have achieved high performance on a wide variety of complex tasks, but the algorithms that they implement are notoriously difficult to interpret. It is often…
Instilling Inductive Biases with Subnetworks
Enyan Zhang, Michael A. Lepori, Ellie Pavlick
Despite the recent success of artificial neural networks on a variety of tasks, we have little knowledge or control over the exact solutions these models implement. Instilling indu…
Deep Neural Networks Can Learn Generalizable Same-Different Visual Relations
Alexa R. Tartaglini, Sheridan Feucht, Michael A. Lepori +4
Although deep neural networks can achieve human-level performance on many object recognition benchmarks, prior work suggests that these same models fail to learn simple abstract re…
NeuroSurgeon: A Toolkit for Subnetwork Analysis
Michael A. Lepori, Ellie Pavlick, Thomas Serre
Despite recent advances in the field of explainability, much remains unknown about the algorithms that neural networks learn to represent. Recent work has attempted to understand t…
Break It Down: Evidence for Structural Compositionality in Neural Networks
Michael A. Lepori, Thomas Serre, Ellie Pavlick
Though modern neural networks have achieved impressive performance in both vision and language tasks, we know little about the functions that they implement. One possibility is tha…
Noise2Astro: Astronomical Image Denoising With Self-Supervised NeuralNetworks
Yunchong Zhang, Brian Nord, Amanda Pagul +1
In observational astronomy, noise obscures signals of interest. Large-scale astronomical surveys are growing in size and complexity, which will produce more data and increase the w…
Unequal Representations: Analyzing Intersectional Biases in Word Embeddings Using Representational Similarity Analysis
Michael A. Lepori
We present a new approach for detecting human-like social biases in word embeddings using representational similarity analysis. Specifically, we probe contextualized and non-contex…
Picking BERT's Brain: Probing for Linguistic Dependencies in Contextualized Embeddings Using Representational Similarity Analysis
Michael A. Lepori, R. Thomas McCoy
As the name implies, contextualized representations of language are typically motivated by their ability to encode context. Which aspects of context are captured by such representa…
Representations of Syntax [MASK] Useful: Effects of Constituency and Dependency Structure in Recursive LSTMs
Michael A. Lepori, Tal Linzen, R. Thomas McCoy
Sequence-based neural networks show significant sensitivity to syntactic structure, but they still perform less well on syntactic tasks than tree-based networks. Such tree-based ne…
Can you hear me ? Sensitive comparisons of human and machine perception
Michael A Lepori, Chaz Firestone
The rise of machine-learning systems that process sensory input has brought with it a rise in comparisons between human and machine perception. But such comparisons face a challeng…