42 citations · 99 across the 9 of their papers we have counts for
10 papers
Scaling Autoregressive Multi-Modal Models: Pretraining and Instruction Tuning
Lili Yu, Bowen Shi, Ramakanth Pasunuru +24
We present CM3Leon (pronounced "Chameleon"), a retrieval-augmented, token-based, decoder-only multi-modal language model capable of generating and infilling both text and images. C…
FACET: Fairness in Computer Vision Evaluation Benchmark
Laura Gustafson, Chloe Rolland, Nikhila Ravi +5
Computer vision models have known performance disparities across attributes such as gender and skin tone. This means during tasks such as classification and detection, model perfor…
DIG In: Evaluating Disparities in Image Generations with Indicators for Geographic Diversity
Melissa Hall, Candace Ross, Adina Williams +3
The unprecedented photorealistic results achieved by recent text-to-image generative systems and their increasing use as plug-and-play content creation solutions make it crucial to…
Towards Reliable Assessments of Demographic Disparities in Multi-Label Image Classifiers
Melissa Hall, Bobbie Chern, Laura Gustafson +4
Disaggregated performance metrics across demographic groups are a hallmark of fairness assessments in computer vision. These metrics successfully incentivized performance improveme…
Vision-Language Models Performing Zero-Shot Tasks Exhibit Gender-based Disparities
Melissa Hall, Laura Gustafson, Aaron Adcock +2
We explore the extent to which zero-shot vision-language models exhibit gender bias for different vision tasks. Vision models traditionally required task-specific labels for repres…
Perturbation Augmentation for Fairer NLP
Rebecca Qian, Candace Ross, Jude Fernandes +3
Unwanted and often harmful social biases are becoming ever more salient in NLP research, affecting both models and datasets. In this work, we ask whether training on demographicall…
Winoground: Probing Vision and Language Models for Visio-Linguistic Compositionality
Tristan Thrush, Ryan Jiang, Max Bartolo +4
We present a novel task and dataset for evaluating the ability of vision and language models to conduct visio-linguistic compositional reasoning, which we call Winoground. Given tw…
CM3: A Causal Masked Multimodal Model of the Internet
Armen Aghajanyan, Bernie Huang, Candace Ross +8
We introduce CM3, a family of causally masked generative models trained over a large corpus of structured multi-modal documents that can contain both text and image tokens. Our new…
Learning a natural-language to LTL executable semantic parser for grounded robotics
Christopher Wang, Candace Ross, Yen-Ling Kuo +2
Children acquire their native language with apparent ease by observing how language is used in context and attempting to use it themselves. They do so without laborious annotations…
Measuring Social Biases in Grounded Vision and Language Embeddings
Candace Ross, Boris Katz, Andrei Barbu
We generalize the notion of social biases from language embeddings to grounded vision and language embeddings. Biases are present in grounded embeddings, and indeed seem to be equa…