18 citations · 22 across the 9 of their papers we have counts for
6 papers · 1 filter
Line of Sight: On Linear Representations in VLLMs
Achyuta Rajaram, Sarah Schwettmann, Jacob Andreas +1
Language models can be equipped with multimodal capabilities by fine-tuning on embeddings of visual inputs. But how do such multimodal models represent images in their hidden activ…
Nearest Neighbor Normalization Improves Multimodal Retrieval
Neil Chowdhury, Franklin Wang, Sumedh Shenoy +3
Multimodal models leverage large-scale pre-training to achieve strong but still imperfect performance on tasks such as image captioning, visual question answering, and cross-modal…
Automatic Discovery of Visual Circuits
Achyuta Rajaram, Neil Chowdhury, Antonio Torralba +2
To date, most discoveries of network subcomponents that implement human-interpretable computations in deep vision models have involved close study of single units and large amounts…
Multimodal Neurons in Pretrained Text-Only Transformers
Sarah Schwettmann, Neil Chowdhury, Samuel Klein +2
Language models demonstrate remarkable capacity to generalize representations learned in one modality to downstream tasks in other modalities. Can we trace this ability to individu…
Natural Language Descriptions of Deep Visual Features
Evan Hernandez, Sarah Schwettmann, David Bau +3
Some neurons in deep networks specialize in recognizing highly specific perceptual, structural, or semantic features of inputs. In computer vision, techniques exist for identifying…
Toward a Visual Concept Vocabulary for GAN Latent Space
Sarah Schwettmann, Evan Hernandez, David Bau +3
A large body of recent work has identified transformations in the latent spaces of generative adversarial networks (GANs) that consistently and interpretably transform generated im…