25 citations · 56 across the 25 of their papers we have counts for
30 papers
InfSplign: Inference-Time Spatial Alignment of Text-to-Image Diffusion Models
Sarah Rastegar, Violeta Chatalbasheva, Sieger Falkena +5
Text-to-image (T2I) diffusion models generate high-quality images but often fail to capture the spatial relations specified in text prompts. This limitation can be traced to two fa…
Side Effects of Erasing Concepts from Diffusion Models
Shaswati Saha, Sourajit Saha, Manas Gaur +1
Concerns about text-to-image (T2I) generative models infringing on privacy, copyright, and safety have led to the development of concept erasure techniques (CETs). The goal of an e…
VOILA: Evaluation of MLLMs For Perceptual Understanding and Analogical Reasoning
Nilay Yilmaz, Maitreya Patel, Yiran Lawrence Luo +4
Multimodal Large Language Models (MLLMs) have become a powerful tool for integrating visual and textual information. Despite their exceptional performance on visual understanding b…
TripletCLIP: Improving Compositional Reasoning of CLIP via Synthetic Vision-Language Negatives
Maitreya Patel, Abhiram Kusumba, Sheng Cheng +4
Contrastive Language-Image Pretraining (CLIP) models maximize the mutual information between text and visual modalities to learn representations. This makes the nature of the train…
REVISION: Rendering Tools Enable Spatial Fidelity in Vision-Language Models
Agneet Chatterjee, Yiran Luo, Tejas Gokhale +2
Text-to-Image (T2I) and multimodal large language models (MLLMs) have been adopted in solutions for several computer vision and multimodal learning tasks. However, it has been foun…
Grounding Stylistic Domain Generalization with Quantitative Domain Shift Measures and Synthetic Scene Images
Yiran Luo, Joshua Feinglass, Tejas Gokhale +3
Domain Generalization (DG) is a challenging task in machine learning that requires a coherent ability to comprehend shifts across various domains through extraction of domain-invar…
On the Robustness of Language Guidance for Low-Level Vision Tasks: Findings from Depth Estimation
Agneet Chatterjee, Tejas Gokhale, Chitta Baral +1
Recent advances in monocular depth estimation have been made by incorporating natural language as additional guidance. Although yielding impressive results, the impact of the langu…
Improving Shift Invariance in Convolutional Neural Networks with Translation Invariant Polyphase Sampling
Sourajit Saha, Tejas Gokhale
Downsampling operators break the shift invariance of convolutional neural networks (CNNs) and this affects the robustness of features learned by CNNs when dealing with even small p…
Getting it Right: Improving Spatial Consistency in Text-to-Image Models
Agneet Chatterjee, Gabriela Ben Melech Stan, Estelle Aflalo +8
One of the key shortcomings in current text-to-image (T2I) models is their inability to consistently generate images which faithfully follow the spatial relationships specified in…
Adversarial Bayesian Augmentation for Single-Source Domain Generalization
Sheng Cheng, Tejas Gokhale, Yezhou Yang
Generalizing to unseen image domains is a challenging problem primarily due to the lack of diverse training data, inaccessible target data, and the large domain shift that may exis…
ConceptBed: Evaluating Concept Learning Abilities of Text-to-Image Diffusion Models
Maitreya Patel, Tejas Gokhale, Chitta Baral +1
The ability to understand visual concepts and replicate and compose these concepts from images is a central goal for computer vision. Recent advances in text-to-image (T2I) models…
End-to-end Knowledge Retrieval with Multi-modal Queries
Man Luo, Zhiyuan Fang, Tejas Gokhale +2
We investigate knowledge retrieval with multi-modal queries, i.e. queries containing information split across image and text inputs, a challenging task that differs from previous w…
Mole Recruitment: Poisoning of Image Classifiers via Selective Batch Sampling
Ethan Wisdom, Tejas Gokhale, Chaowei Xiao +1
In this work, we present a data poisoning attack that confounds machine learning models without any manipulation of the image or label. This is achieved by simply leveraging the mo…
Benchmarking Spatial Relationships in Text-to-Image Generation
Tejas Gokhale, Hamid Palangi, Besmira Nushi +5
Spatial understanding is a fundamental aspect of computer vision and integral for human-level reasoning about images, making it an important component for grounded language underst…
CRIPP-VQA: Counterfactual Reasoning about Implicit Physical Properties via Video Question Answering
Maitreya Patel, Tejas Gokhale, Chitta Baral +1
Videos often capture objects, their visible properties, their motion, and the interactions between different objects. Objects also have physical properties such as mass, which the…
Improving Diversity with Adversarially Learned Transformations for Domain Generalization
Tejas Gokhale, Rushil Anirudh, Jayaraman J. Thiagarajan +3
To be successful in single source domain generalization, maximizing diversity of synthesized domains has emerged as one of the most effective strategies. Many of the recent success…
To Find Waldo You Need Contextual Cues: Debiasing Who's Waldo
Yiran Luo, Pratyay Banerjee, Tejas Gokhale +2
We present a debiased dataset for the Person-centric Visual Grounding (PCVG) task first proposed by Cui et al. (2021) in the Who's Waldo dataset. Given an image and a caption, PCVG…
Generalized but not Robust? Comparing the Effects of Data Modification Methods on Out-of-Domain Generalization and Adversarial Robustness
Tejas Gokhale, Swaroop Mishra, Man Luo +2
Data modification, either via additional training datasets, data augmentation, debiasing, and dataset filtering, has been proposed as an effective solution for generalizing to out-…
Improving Biomedical Information Retrieval with Neural Retrievers
Man Luo, Arindam Mitra, Tejas Gokhale +1
Information retrieval (IR) is essential in search engines and dialogue systems as well as natural language processing tasks such as open-domain question answering. IR serve an impo…
Unsupervised Natural Language Inference Using PHL Triplet Generation
Neeraj Varshney, Pratyay Banerjee, Tejas Gokhale +1
Transformer-based models achieve impressive performance on numerous Natural Language Inference (NLI) benchmarks when trained on respective training datasets. However, in certain ca…
Semantically Distributed Robust Optimization for Vision-and-Language Inference
Tejas Gokhale, Abhishek Chaudhary, Pratyay Banerjee +2
Analysis of vision-and-language models has revealed their brittleness under linguistic phenomena such as paraphrasing, negation, textual entailment, and word substitutions with syn…
Weakly Supervised Relative Spatial Reasoning for Visual Question Answering
Pratyay Banerjee, Tejas Gokhale, Yezhou Yang +1
Vision-and-language (V\&L) reasoning necessitates perception of visual concepts such as objects and actions, understanding semantics and language grounding, and reasoning about the…
Self-Supervised Test-Time Learning for Reading Comprehension
Pratyay Banerjee, Tejas Gokhale, Chitta Baral
Recent work on unsupervised question answering has shown that models can be trained with procedurally generated question-answer pairs and can achieve performance competitive with s…
WeaQA: Weak Supervision via Captions for Visual Question Answering
Pratyay Banerjee, Tejas Gokhale, Yezhou Yang +1
Methodologies for training visual question answering (VQA) models assume the availability of datasets with human-annotated \textit{Image-Question-Answer} (I-Q-A) triplets. This has…