1 citations · 2 across the 8 of their papers we have counts for
11 papers
SPELUNKA: A Framework for Finding Unanticipated Associations in Vision-Language Models
Cassandra Parent, Isha Agarwal, Kumail Alhamoud +4
Vision-Language Models (VLMs) are the backbone of many state-of-the-art multimodal systems, but often encode spurious and harmful associations from their training data. While prior…
Sense and Sensitivity: Benchmarking LLM Clinical Triage Recommendations with Physician Experts
Abinitha Gourabathina, Haoran Zhang, Yuexing Hao +2
As large language models (LLMs) are increasingly used in clinical settings, it is critical to evaluate their reliability under realistic variation in clinical text. We study this q…
Complementing Self-Consistency with Cross-Model Disagreement for Uncertainty Quantification
Kimia Hamidieh, Veronika Thost, Walter Gerych +2
Large language models (LLMs) often produce confident yet incorrect responses, and uncertainty quantification is one potential solution to more robust usage. Recent works routinely…
An Investigation of Memorization Risk in Healthcare Foundation Models
Sana Tonekaboni, Lena Stempfle, Adibvafa Fallahpour +2
Foundation models trained on large-scale de-identified electronic health records (EHRs) hold promise for clinical applications. However, their capacity to memorize patient informat…
The MedPerturb Dataset: What Non-Content Perturbations Reveal About Human and Clinical LLM Decision Making
Abinitha Gourabathina, Yuexing Hao, Walter Gerych +1
Clinical robustness is critical to the safe deployment of medical Large Language Models (LLMs), but key questions remain about how LLMs and humans may differ in response to the rea…
When Style Breaks Safety: Defending LLMs Against Superficial Style Alignment
Yuxin Xiao, Sana Tonekaboni, Walter Gerych +2
Large language models (LLMs) can be prompted with specific styles (e.g., formatting responses as lists), including in malicious queries. Prior jailbreak research mainly augments th…
MaskMedPaint: Masked Medical Image Inpainting with Diffusion Models for Mitigation of Spurious Correlations
Qixuan Jin, Walter Gerych, Marzyeh Ghassemi
Spurious features associated with class labels can lead image classifiers to rely on shortcuts that don't generalize well to new domains. This is especially problematic in medical…
BendVLM: Test-Time Debiasing of Vision-Language Embeddings
Walter Gerych, Haoran Zhang, Kimia Hamidieh +4
Vision-language model (VLM) embeddings have been shown to encode biases present in their training data, such as societal biases that prescribe negative characteristics to members o…
Identifying Implicit Social Biases in Vision-Language Models
Kimia Hamidieh, Haoran Zhang, Walter Gerych +2
Vision-language models, like CLIP (Contrastive Language Image Pretraining), are becoming increasingly popular for a wide range of multimodal retrieval tasks. However, prior work ha…
TAXI: Evaluating Categorical Knowledge Editing for Language Models
Derek Powell, Walter Gerych, Thomas Hartvigsen
Humans rarely learn one fact in isolation. Instead, learning a new fact induces knowledge of other facts about the world. For example, in learning a korat is a type of cat, you als…
Learning under Temporal Label Noise
Sujay Nagaraj, Walter Gerych, Sana Tonekaboni +3
Many time series classification tasks, where labels vary over time, are affected by label noise that also varies over time. Such noise can cause label quality to improve, worsen, o…