8 papers
Attributes Should Come from Images, Not Class Names: Distribution-Conditioned Attribute Selection for Vision-Language Models
Gautam Rajendrakumar Gare, Jia Shi, Zhiqiu Lin +3
A popular route to interpretable zero-shot classification asks a large language model (LLM) to describe each class name and prompts CLIP with the resulting descriptors. We show tha…
Ask Twice, Look Twice: Prompt Echoing Resolves the Question-First Paradox in Vision-Language Models
Rakshanda Hassan Abhinandan, John Galeotti, Deva Ramanan +1
Where should the question go in a vision-language model (VLM) prompt: before the image or after it? Intuition says before: knowing what is asked should tell the model where to look…
DetPO: In-Context Learning with Multi-Modal LLMs for Few-Shot Object Detection
Gautam Rajendrakumar Gare, Neehar Peri, Matvei Popov +3
Multi-Modal LLMs (MLLMs) demonstrate strong visual grounding capabilities on popular object detection benchmarks like OdinW-13 and RefCOCO. However, state-of-the-art models still s…
Do Real-World Datasets Contain Natural Experiments? An Empirical Study Using Causal Feature Selection
Gautam Gare, John Galeotti, Michael Mozer +2
In nature, events that affect some individuals or groups but not others constitute an implicit intervention and are known as natural experiments. For example, the COVID-19 pandemic…
LEARNER: Contrastive Pretraining for Learning Fine-Grained Patient Progression from Coarse Inter-Patient Labels
Jana Armouti, Nikhil Madaan, Rohan Panda +8
Predicting whether a treatment leads to meaningful improvement is a central challenge in personalized medicine, particularly when disease progression manifests as subtle visual cha…
Label Uncertainty for Ultrasound Segmentation
Malini Shivaram, Gautam Rajendrakumar Gare, Laura Hutchins +11
In medical imaging, inter-observer variability among radiologists often introduces label uncertainty, particularly in modalities where visual interpretation is subjective. Lung ult…