565 citations · 1.1k across the 53 of their papers we have counts for
8 papers · 1 filter
Beyond the Doors of Perception: Vision Transformers Represent Relations Between Objects
Michael A. Lepori, Alexa R. Tartaglini, Wai Keen Vong +3
Though vision transformers (ViTs) have achieved state-of-the-art performance in a variety of settings, they exhibit surprising failures when performing tasks involving visual relat…
Analyzing Modular Approaches for Visual Question Decomposition
Apoorv Khandelwal, Ellie Pavlick, Chen Sun
Modular neural networks without additional training have recently been shown to surpass end-to-end neural networks on challenging vision-language tasks. The latest such methods sim…
Deep Neural Networks Can Learn Generalizable Same-Different Visual Relations
Alexa R. Tartaglini, Sheridan Feucht, Michael A. Lepori +4
Although deep neural networks can achieve human-level performance on many object recognition benchmarks, prior work suggests that these same models fail to learn simple abstract re…
Training Priors Predict Text-To-Image Model Performance
Charles Lovering, Ellie Pavlick
Text-to-image models can often generate some relations, i.e., "astronaut riding horse", but fail to generate other relations composed of the same basic parts, i.e., "horse riding a…
Comparing Trajectory and Vision Modalities for Verb Representation
Dylan Ebert, Chen Sun, Ellie Pavlick
Three-dimensional trajectories, or the 3D position and rotation of objects over time, have been shown to encode key aspects of verb semantics (e.g., the meanings of roll vs. slide)…
Does CLIP Bind Concepts? Probing Compositionality in Large Image Models
Martha Lewis, Nihal V. Nayak, Peilin Yu +4
Large-scale neural network models combining text and images have made incredible progress in recent years. However, it remains an open question to what extent such models encode co…
Do Vision-Language Pretrained Models Learn Composable Primitive Concepts?
Tian Yun, Usha Bhalla, Ellie Pavlick +1
Vision-language (VL) pretrained models have achieved impressive performance on multimodal reasoning and zero-shot recognition tasks. Many of these VL models are pretrained on unlab…
Can Language Models Encode Perceptual Structure Without Grounding? A Case Study in Color
Mostafa Abdou, Artur Kulmizev, Daniel Hershcovich +3
Pretrained language models have been shown to encode relational information, such as the relations between entities or concepts in knowledge-bases -- (Paris, Capital, France). Howe…