38 citations · 159 across the 20 of their papers we have counts for
23 papers
Trust but Verify: Programmatic VLM Evaluation in the Wild
Viraj Prabhu, Senthil Purushwalkam, An Yan +2
Vision-Language Models (VLMs) often generate plausible but incorrect responses to visual queries. However, reliably quantifying the effect of such hallucinations in free-form respo…
xGen-MM (BLIP-3): A Family of Open Large Multimodal Models
Le Xue, Manli Shu, Anas Awadalla +30
This paper introduces BLIP-3, an open framework for developing Large Multimodal Models (LMMs). The framework comprises meticulously curated datasets, a training recipe, model archi…
We're Not Using Videos Effectively: An Updated Domain Adaptive Video Segmentation Baseline
Simar Kareer, Vivek Vijaykumar, Harsh Maheshwari +3
There has been abundant work in unsupervised domain adaptation for semantic segmentation (DAS) seeking to adapt a model trained on images from a labeled source domain to an unlabel…
AUGCAL: Improving Sim2Real Adaptation by Uncertainty Calibration on Augmented Synthetic Images
Prithvijit Chattopadhyay, Bharat Goyal, Boglarka Ecsedi +2
Synthetic data (SIM) drawn from simulators have emerged as a popular alternative for training models where acquiring annotated real-world images is difficult. However, transferring…
Battle of the Backbones: A Large-Scale Comparison of Pretrained Models across Computer Vision Tasks
Micah Goldblum, Hossein Souri, Renkun Ni +10
Neural network based computer vision systems are typically built on a backbone, a pretrained or randomly initialized feature extractor. Several years ago, the default option was an…
FACTS: First Amplify Correlations and Then Slice to Discover Bias
Sriram Yenamandra, Pratik Ramesh, Viraj Prabhu +1
Computer vision datasets frequently contain spurious correlations between task-relevant labels and (easy to learn) latent task-irrelevant attributes (e.g. context). Models trained…
ICON: Reliably Benchmarking Predictive Inequity in Object Detection
Sruthi Sudhakar, Viraj Prabhu, Olga Russakovsky +1
As computer vision systems are being increasingly deployed at scale in high-stakes applications like autonomous driving, concerns about social bias in these systems are rising. Ana…
LANCE: Stress-testing Visual Models by Generating Language-guided Counterfactual Images
Viraj Prabhu, Sriram Yenamandra, Prithvijit Chattopadhyay +1
We propose an automated algorithm to stress-test a trained visual model by generating language-guided counterfactual test images (LANCE). Our method leverages recent progress in la…
Bridging the Sim2Real gap with CARE: Supervised Detection Adaptation with Conditional Alignment and Reweighting
Viraj Prabhu, David Acuna, Andrew Liao +5
Sim2Real domain adaptation (DA) research focuses on the constrained setting of adapting from a labeled synthetic source domain to an unlabeled or sparsely labeled real target domai…
Mitigating Bias in Visual Transformers via Targeted Alignment
Sruthi Sudhakar, Viraj Prabhu, Arvindkumar Krishnakumar +1
As transformer architectures become increasingly prevalent in computer vision, it is critical to understand their fairness implications. We perform the first study of the fairness…
Adapting Self-Supervised Vision Transformers by Probing Attention-Conditioned Masking Consistency
Viraj Prabhu, Sriram Yenamandra, Aaditya Singh +1
Visual domain adaptation (DA) seeks to transfer trained models to unseen, unlabeled domains across distribution shift, but approaches typically focus on adapting convolutional neur…
Can domain adaptation make object recognition work for everyone?
Viraj Prabhu, Ramprasaath R. Selvaraju, Judy Hoffman +1
Despite the rapid progress in deep visual recognition, modern computer vision datasets significantly overrepresent the developed world and models trained on such datasets underperf…
UDIS: Unsupervised Discovery of Bias in Deep Visual Recognition Models
Arvindkumar Krishnakumar, Viraj Prabhu, Sruthi Sudhakar +1
Deep learning models have been shown to learn spurious correlations from data that sometimes lead to systematic failures for certain subpopulations. Prior work has typically diagno…
AUGCO: Augmentation Consistency-guided Self-training for Source-free Domain Adaptive Semantic Segmentation
Viraj Prabhu, Shivam Khare, Deeksha Kartik +1
Most modern approaches for domain adaptive semantic segmentation rely on continued access to source data during adaptation, which may be infeasible due to computational or privacy…
SENTRY: Selective Entropy Optimization via Committee Consistency for Unsupervised Domain Adaptation
Viraj Prabhu, Shivam Khare, Deeksha Kartik +1
Many existing approaches for unsupervised domain adaptation (UDA) focus on adapting under only data distribution shift and offer limited success under additional cross-domain label…
Active Domain Adaptation via Clustering Uncertainty-weighted Embeddings
Viraj Prabhu, Arjun Chandrasekaran, Kate Saenko +1
Generalizing deep neural networks to new target domains is critical to their real-world utility. In practice, it may be feasible to get some target data labeled, but to be cost-eff…
Open Set Medical Diagnosis
Viraj Prabhu, Anitha Kannan, Geoffrey J. Tso +4
Machine-learned diagnosis models have shown promise as medical aides but are trained under a closed-set assumption, i.e. that models will only encounter conditions on which they ha…
Prototypical Clustering Networks for Dermatological Disease Diagnosis
Viraj Prabhu, Anitha Kannan, Murali Ravuri +3
We consider the problem of image classification for the purpose of aiding doctors in dermatological diagnosis. Dermatological diagnosis poses two major challenges for standard off-…
Do Explanations make VQA Models more Predictable to a Human?
Arjun Chandrasekaran, Viraj Prabhu, Deshraj Yadav +2
A rich line of research attempts to make deep neural networks more transparent by generating human-interpretable 'explanations' of their decision process, especially for interactiv…
Fabrik: An Online Collaborative Neural Network Editor
Utsav Garg, Viraj Prabhu, Deshraj Yadav +3
We present Fabrik, an online neural network editor that provides tools to visualize, edit, and share neural networks from within a browser. Fabrik provides a simple and intuitive G…
Evaluating Visual Conversational Agents via Cooperative Human-AI Games
Prithvijit Chattopadhyay, Deshraj Yadav, Viraj Prabhu +5
As AI continues to advance, human-AI teams are inevitable. However, progress in AI is routinely measured in isolation, without a human in the loop. It is crucial to benchmark progr…
The Promise of Premise: Harnessing Question Premises in Visual Question Answering
Aroma Mahendru, Viraj Prabhu, Akrit Mohapatra +2
In this paper, we make a simple observation that questions about images often contain premises - objects and relationships implied by the question - and that reasoning about premis…
It Takes Two to Tango: Towards Theory of AI's Mind
Arjun Chandrasekaran, Deshraj Yadav, Prithvijit Chattopadhyay +2
Theory of Mind is the ability to attribute mental states (beliefs, intents, knowledge, perspectives, etc.) to others and recognize that these mental states may differ from one's ow…