90 citations · 205 across the 22 of their papers we have counts for
14 papers · 1 filter
Symbal: Detecting Systematic Misalignments in Model-Generated Captions
Maya Varma, Jean-Benoit Delbrouck, Sophie Ostmeier +2
Multimodal large language models (MLLMs) often introduce errors when generating image captions, resulting in misaligned image-text pairs. Our work focuses on a class of captioning…
CheXmix: Unified Generative Pretraining for Vision Language Models in Medical Imaging
Ashwin Kumar, Robbie Holland, Corey Barrett +8
Recent medical multimodal foundation models are built as multimodal LLMs (MLLMs) by connecting a CLIP-pretrained vision encoder to an LLM using LLaVA-style finetuning. This two-sta…
A Reasoning-Enabled Vision-Language Foundation Model for Chest X-ray Interpretation
Yabin Zhang, Chong Wang, Yunhe Gao +19
Chest X-rays (CXRs) are among the most frequently performed imaging examinations worldwide, yet rising imaging volumes increase radiologist workload and the risk of diagnostic erro…
Activation Matters: Test-time Activated Negative Labels for OOD Detection with Vision-Language Models
Yabin Zhang, Maya Varma, Yunhe Gao +4
Out-of-distribution (OOD) detection aims to identify samples that deviate from in-distribution (ID). One popular pipeline addresses this by introducing negative labels distant from…
Learning Generalizable 3D Medical Image Representations from Mask-Guided Self-Supervision
Yunhe Gao, Yabin Zhang, Chong Wang +5
Foundation models have transformed vision and language by learning general-purpose representations from large-scale unlabeled data, yet 3D medical imaging lacks analogous approache…
A data- and compute-efficient chest X-ray foundation model beyond aggressive scaling
Chong Wang, Yabin Zhang, Yunhe Gao +9
Foundation models for medical imaging are typically pretrained on increasingly large datasets, following a "scale-at-all-costs" paradigm. However, this strategy faces two critical…
RadDiff: Describing Differences in Radiology Image Sets with Natural Language
Xiaoxian Shen, Yuhui Zhang, Sahithi Ankireddy +5
Understanding how two radiology image sets differ is critical for generating clinical insights and for interpreting medical AI systems. We introduce RadDiff, a multimodal agentic s…
TRoVe: Discovering Error-Inducing Static Feature Biases in Temporal Vision-Language Models
Maya Varma, Jean-Benoit Delbrouck, Sophie Ostmeier +2
Vision-language models (VLMs) have made great strides in addressing temporal understanding tasks, which involve characterizing visual changes across a sequence of images. However,…
From Detection to Mitigation: Addressing Bias in Deep Learning Models for Chest X-Ray Diagnosis
Clemence Mottez, Louisa Fay, Maya Varma +2
Deep learning models have shown promise in improving diagnostic accuracy from chest X-rays, but they also risk perpetuating healthcare disparities when performance varies across de…
RaVL: Discovering and Mitigating Spurious Correlations in Fine-Tuned Vision-Language Models
Maya Varma, Jean-Benoit Delbrouck, Zhihong Chen +2
Fine-tuned vision-language models (VLMs) often capture spurious correlations between image features and textual attributes, resulting in degraded zero-shot performance at test time…
CheXalign: Preference fine-tuning in chest X-ray interpretation models without human feedback
Dennis Hein, Zhihong Chen, Sophie Ostmeier +8
Radiologists play a crucial role in translating medical images into actionable reports. However, the field faces staffing shortages and increasing workloads. While automated approa…
LieRE: Lie Rotational Positional Encodings
Sophie Ostmeier, Brian Axelrod, Maya Varma +3
Transformer architectures rely on position encodings to model the spatial structure of input data. Rotary Position Encoding (RoPE) is a widely used method in language models that e…
Merlin: A Computed Tomography Vision-Language Foundation Model and Dataset
Louis Blankemeier, Ashwin Kumar, Joseph Paul Cohen +37
The large volume of abdominal computed tomography (CT) scans coupled with the shortage of radiologists have intensified the need for automated medical image analysis tools. Previou…
A Vision-Language Foundation Model to Enhance Efficiency of Chest X-ray Interpretation
Zhihong Chen, Maya Varma, Justin Xu +20
Over 1.4 billion chest X-rays (CXRs) are performed annually due to their cost-effectiveness as an initial diagnostic test. This scale of radiological studies provides a significant…