4 citations · 12 across the 20 of their papers we have counts for
23 papers
Sense and Sensitivity: Benchmarking LLM Clinical Triage Recommendations with Physician Experts
Abinitha Gourabathina, Haoran Zhang, Yuexing Hao +2
As large language models (LLMs) are increasingly used in clinical settings, it is critical to evaluate their reliability under realistic variation in clinical text. We study this q…
Rank-Constrained Adaptation for Reliable Real-World Performance
Abinitha Gourabathina, Hyewon Jeong, Teya Bergamaschi +2
Deep learning models trained to optimize average accuracy often exhibit systematic failures on particular subpopulations. In real-world settings like healthcare, the subpopulations…
Explainable AI as a Double-Edged Sword in Dermatology: The Impact on Clinicians versus The Public
Xuhai Xu, Haoyu Hu, Haoran Zhang +21
Artificial intelligence (AI) is increasingly permeating healthcare, from physician assistants to consumer applications. Since AI algorithm's opacity challenges human interaction, e…
Aggregation Hides Out-of-Distribution Generalization Failures from Spurious Correlations
Olawale Salaudeen, Haoran Zhang, Kumail Alhamoud +2
Benchmarks for out-of-distribution (OOD) generalization frequently show a strong positive correlation between in-distribution (ID) and OOD accuracy across models, termed "accuracy-…
The Role of Computing Resources in Publishing Foundation Model Research
Yuexing Hao, Yue Huang, Haoran Zhang +8
Cutting-edge research in Artificial Intelligence (AI) requires considerable resources, including Graphics Processing Units (GPUs), data, and human resources. In this paper, we eval…
An Investigation of Memorization Risk in Healthcare Foundation Models
Sana Tonekaboni, Lena Stempfle, Adibvafa Fallahpour +2
Foundation models trained on large-scale de-identified electronic health records (EHRs) hold promise for clinical applications. However, their capacity to memorize patient informat…
Train Long, Think Short: Curriculum Learning for Efficient Reasoning
Hasan Abed Al Kader Hammoud, Kumail Alhamoud, Abed Hammoud +3
Recent work on enhancing the reasoning abilities of large language models (LLMs) has introduced explicit length control as a means of constraining computational cost while preservi…
The MedPerturb Dataset: What Non-Content Perturbations Reveal About Human and Clinical LLM Decision Making
Abinitha Gourabathina, Yuexing Hao, Walter Gerych +1
Clinical robustness is critical to the safe deployment of medical Large Language Models (LLMs), but key questions remain about how LLMs and humans may differ in response to the rea…
Tiered Agentic Oversight: A Hierarchical Multi-Agent System for Healthcare Safety
Yubin Kim, Hyewon Jeong, Chanwoo Park +9
Large language models (LLMs) deployed as agents introduce significant safety risks in clinical settings due to their potential for error and single points of failure. We introduce…
KScope: A Framework for Characterizing the Knowledge Status of Language Models
Yuxin Xiao, Shan Chen, Jack Gallifant +3
Characterizing a large language model's (LLM's) knowledge of a given question is challenging. As a result, prior work has primarily examined LLM behavior under knowledge conflicts,…
When Style Breaks Safety: Defending LLMs Against Superficial Style Alignment
Yuxin Xiao, Sana Tonekaboni, Walter Gerych +2
Large language models (LLMs) can be prompted with specific styles (e.g., formatting responses as lists), including in malicious queries. Prior jailbreak research mainly augments th…
MedPAIR: Measuring Physicians and AI Relevance Alignment in Medical Question Answering
Yuexing Hao, Kumail Alhamoud, Hyewon Jeong +6
Large Language Models (LLMs) have demonstrated remarkable performance on various medical question-answering (QA) benchmarks, including standardized medical exams. However, correct…
Vision-Language Models Do Not Understand Negation
Kumail Alhamoud, Shaden Alshammari, Yonglong Tian +4
Many practical vision-language applications require models that understand negation, e.g., when using natural language to retrieve images which contain certain objects but not othe…
MaskMedPaint: Masked Medical Image Inpainting with Diffusion Models for Mitigation of Spurious Correlations
Qixuan Jin, Walter Gerych, Marzyeh Ghassemi
Spurious features associated with class labels can lead image classifiers to rely on shortcuts that don't generalize well to new domains. This is especially problematic in medical…
BendVLM: Test-Time Debiasing of Vision-Language Embeddings
Walter Gerych, Haoran Zhang, Kimia Hamidieh +4
Vision-language model (VLM) embeddings have been shown to encode biases present in their training data, such as societal biases that prescribe negative characteristics to members o…
Identifying Implicit Social Biases in Vision-Language Models
Kimia Hamidieh, Haoran Zhang, Walter Gerych +2
Vision-language models, like CLIP (Contrastive Language Image Pretraining), are becoming increasingly popular for a wide range of multimodal retrieval tasks. However, prior work ha…
Personalizing Prostate Cancer Education for Patients Using an EHR-Integrated LLM Agent
Yuexing Hao, Jason Holmes, Mark R. Waddle +10
Cancer patients often lack timely education and personalized support due to clinician workload. This quality improvement study develops and evaluates a Large Language Model (LLM) a…
Measuring Stochastic Data Complexity with Boltzmann Influence Functions
Nathan Ng, Roger Grosse, Marzyeh Ghassemi
Estimating the uncertainty of a model's prediction on a test point is a crucial part of ensuring reliability and calibration under distribution shifts. A minimum description length…
FedMedICL: Towards Holistic Evaluation of Distribution Shifts in Federated Medical Imaging
Kumail Alhamoud, Yasir Ghunaim, Motasem Alfarra +5
For medical imaging AI models to be clinically impactful, they must generalize. However, this goal is hindered by (i) diverse types of distribution shifts, such as temporal, demogr…
LEMoN: Label Error Detection using Multimodal Neighbors
Haoran Zhang, Aparna Balagopalan, Nassim Oufattole +4
Large repositories of image-caption pairs are essential for the development of vision-language models. However, these datasets are often extracted from noisy data scraped from the…
Data Debiasing with Datamodels (D3M): Improving Subgroup Robustness via Data Selection
Saachi Jain, Kimia Hamidieh, Kristian Georgiev +3
Machine learning models can fail on subgroups that are underrepresented during training. While techniques such as dataset balancing can improve performance on underperforming group…
Views Can Be Deceiving: Improved SSL Through Feature Space Augmentation
Kimia Hamidieh, Haoran Zhang, Swami Sankaranarayanan +1
Supervised learning methods have been found to exhibit inductive biases favoring simpler features. When such features are spuriously correlated with the label, this can result in s…
Improving Black-box Robustness with In-Context Rewriting
Kyle O'Brien, Nathan Ng, Isha Puri +5
Machine learning models for text classification often excel on in-distribution (ID) data but struggle with unseen out-of-distribution (OOD) inputs. Most techniques for improving OO…