204 citations · 239 across the 6 of their papers we have counts for
9 papers
Foveated Reasoning: Stateful, Action-based Visual Focusing for Vision-Language Models
Juhong Min, Lazar Valkov, Vitali Petsiuk +2
Vision-language models benefit from high-resolution images, but the increase in visual-token count incurs high compute overhead. Humans resolve this tension via foveation: a coarse…
GoldiCLIP: The Goldilocks Approach for Balancing Explicit Supervision for Language-Image Pretraining
Deen Dayal Mohan, Hossein Souri, Vitali Petsiuk +4
Until recently, the success of large-scale vision-language models (VLMs) has primarily relied on billion-sample datasets, posing a significant barrier to progress. Latest works hav…
ARGENT: Adaptive Hierarchical Image-Text Representations
Chuong Huynh, Hossein Souri, Abhinav Kumar +3
Large-scale Vision-Language Models (VLMs) such as CLIP learn powerful semantic representations but operate in Euclidean space, which fails to capture the inherent hierarchical stru…
Concept Arithmetics for Circumventing Concept Inhibition in Diffusion Models
Vitali Petsiuk, Kate Saenko
Motivated by ethical and legal concerns, the scientific community is actively developing methods to limit the misuse of Text-to-Image diffusion models for reproducing copyrighted,…
Human Evaluation of Text-to-Image Models on a Multi-Task Benchmark
Vitali Petsiuk, Alexander E. Siemenn, Saisamrit Surbehera +11
We provide a new multi-task benchmark for evaluating text-to-image models. We perform a human evaluation comparing the most common open-source (Stable Diffusion) and commercial (DA…
Black-box Explanation of Object Detectors via Saliency Maps
Vitali Petsiuk, Rajiv Jain, Varun Manjunatha +4
We propose D-RISE, a method for generating visual explanations for the predictions of object detectors. Utilizing the proposed similarity metric that accounts for both localization…
Why do These Match? Explaining the Behavior of Image Similarity Models
Bryan A. Plummer, Mariya I. Vasileva, Vitali Petsiuk +2
Explaining a deep learning model can help users understand its behavior and allow researchers to discern its shortcomings. Recent work has primarily focused on explaining models fo…
Guided Zoom: Questioning Network Evidence for Fine-grained Classification
Sarah Adel Bargal, Andrea Zunino, Vitali Petsiuk +4
We propose Guided Zoom, an approach that utilizes spatial grounding of a model's decision to make more informed predictions. It does so by making sure the model has "the right reas…
RISE: Randomized Input Sampling for Explanation of Black-box Models
Vitali Petsiuk, Abir Das, Kate Saenko
Deep neural networks are being used increasingly to automate data analysis and decision making, yet their decision-making process is largely unclear and is difficult to explain to…