92 citations · 263 across the 8 of their papers we have counts for
10 papers
MAPL: Parameter-Efficient Adaptation of Unimodal Pre-Trained Models for Vision-Language Few-Shot Prompting
Oscar Mañas, Pau Rodriguez, Saba Ahmadi +3
Large pre-trained models have proved to be remarkable zero- and (prompt-based) few-shot learners in unimodal vision and language tasks. We propose MAPL, a simple and parameter-effi…
Image Retrieval from Contextual Descriptions
Benno Krojer, Vaibhav Adlakha, Vibhav Vineet +3
The ability to integrate context, including perceptual and temporal cues, plays a pivotal role in grounding the meaning of a linguistic utterance. In order to measure to what exten…
Question-Conditioned Counterfactual Image Generation for VQA
Jingjing Pan, Yash Goyal, Stefan Lee
While Visual Question Answering (VQA) models continue to push the state-of-the-art forward, they largely remain black-boxes - failing to provide insight into how or why an answer i…
Explaining Classifiers with Causal Concept Effect (CaCE)
Yash Goyal, Amir Feder, Uri Shalit +1
How can we understand classification decisions made by deep neural networks? Many existing explainability methods rely solely on correlations and fail to account for confounding, w…
Counterfactual Visual Explanations
Yash Goyal, Ziyan Wu, Jan Ernst +3
In this work, we develop a technique to produce counterfactual visual explanations. Given a 'query' image for which a vision system predicts class , a counterfactual visual…
Making the V in VQA Matter: Elevating the Role of Image Understanding in Visual Question Answering
Yash Goyal, Tejas Khot, Douglas Summers-Stay +2
Problems at the intersection of vision and language are of significant importance both as challenging research questions and for the rich set of applications they enable. However,…
Towards Transparent AI Systems: Interpreting Visual Question Answering Models
Yash Goyal, Akrit Mohapatra, Devi Parikh +1
Deep neural networks have shown striking progress and obtained state-of-the-art results in many AI research fields in the recent years. However, it is often unsatisfying to not kno…
Resolving Language and Vision Ambiguities Together: Joint Segmentation & Prepositional Attachment Resolution in Captioned Scenes
Gordon Christie, Ankit Laddha, Aishwarya Agrawal +4
We present an approach to simultaneously perform semantic segmentation and prepositional phrase attachment resolution for captioned images. Some ambiguities in language cannot be r…
Yin and Yang: Balancing and Answering Binary Visual Questions
Peng Zhang, Yash Goyal, Douglas Summers-Stay +2
The complex compositional structure of language makes problems at the intersection of vision and language challenging. But language also provides a strong prior that can result in…
CloudCV: Large Scale Distributed Computer Vision as a Cloud Service
Harsh Agrawal, Clint Solomon Mathialagan, Yash Goyal +5
We are witnessing a proliferation of massive visual data. Unfortunately scaling existing computer vision algorithms to large datasets leaves researchers repeatedly solving the same…