18 citations · 64 across the 23 of their papers we have counts for
11 papers · 1 filter
SocialCounterfactuals: Probing and Mitigating Intersectional Social Biases in Vision-Language Models with Counterfactual Examples
Phillip Howard, Avinash Madasu, Tiep Le +3
While vision-language models (VLMs) have achieved remarkable performance improvements recently, there is growing evidence that these models also posses harmful biases with respect…
Analyzing Zero-Shot Abilities of Vision-Language Models on Video Understanding Tasks
Avinash Madasu, Anahita Bhiwandiwalla, Vasudev Lal
Foundational multimodal models pre-trained on large scale image-text pairs or video-text pairs or both have shown strong generalization abilities on downstream tasks. However unlik…
NeuroPrompts: An Adaptive Framework to Optimize Prompts for Text-to-Image Generation
Shachar Rosenman, Vasudev Lal, Phillip Howard
Despite impressive recent advances in text-to-image diffusion models, obtaining high-quality images often requires prompt engineering by humans who have developed expertise in usin…
COCO-Counterfactuals: Automatically Constructed Counterfactual Examples for Image-Text Pairs
Tiep Le, Vasudev Lal, Phillip Howard
Counterfactual examples have proven to be valuable in the field of natural language processing (NLP) for both evaluating and improving the robustness of language models to spurious…
Probing Intersectional Biases in Vision-Language Models with Counterfactual Examples
Phillip Howard, Avinash Madasu, Tiep Le +2
While vision-language models (VLMs) have achieved remarkable performance improvements recently, there is growing evidence that these models also posses harmful biases with respect…
ICSVR: Investigating Compositional and Syntactic Understanding in Video Retrieval Models
Avinash Madasu, Vasudev Lal
Video retrieval (VR) involves retrieving the ground truth video from the video database given a text caption or vice-versa. The two important components of compositionality: object…
ManagerTower: Aggregating the Insights of Uni-Modal Experts for Vision-Language Representation Learning
Xiao Xu, Bei Li, Chenfei Wu +6
Two-Tower Vision-Language (VL) models have shown promising improvements on various downstream VL tasks. Although the most advanced work improves performance by building bridges bet…
Brain encoding models based on multimodal transformers can transfer across language and vision
Jerry Tang, Meng Du, Vy A. Vo +2
Encoding models have been used to assess how the human brain represents concepts in language and vision. While language and vision rely on similar concept representations, current…
NeuroComparatives: Neuro-Symbolic Distillation of Comparative Knowledge
Phillip Howard, Junlin Wang, Vasudev Lal +3
Comparative knowledge (e.g., steel is stronger and heavier than styrofoam) is an essential component of our world knowledge, yet understudied in prior literature. In this paper, we…
Thrill-K Architecture: Towards a Solution to the Problem of Knowledge Based Understanding
Gadi Singer, Joscha Bach, Tetiana Grinberg +4
While end-to-end learning systems are rapidly gaining capabilities and popularity, the increasing computational demands for deploying such systems, along with a lack of flexibility…
Is Multimodal Vision Supervision Beneficial to Language?
Avinash Madasu, Vasudev Lal
Vision (image and video) - Language (VL) pre-training is the recent popular paradigm that achieved state-of-the-art results on multi-modal tasks like image-retrieval, video-retriev…