20 citations · 33 across the 14 of their papers we have counts for
16 papers
KScope: A Framework for Characterizing the Knowledge Status of Language Models
Yuxin Xiao, Shan Chen, Jack Gallifant +3
Characterizing a large language model's (LLM's) knowledge of a given question is challenging. As a result, prior work has primarily examined LLM behavior under knowledge conflicts,…
MedBrowseComp: Benchmarking Medical Deep Research and Computer Use
Shan Chen, Pedro Moreira, Yuxin Xiao +6
Large language models (LLMs) are increasingly envisioned as decision-support tools in clinical practice, yet safe clinical reasoning demands integrating heterogeneous knowledge bas…
Sparse Autoencoder Features for Classifications and Transferability
Jack Gallifant, Shan Chen, Kuleen Sasse +3
Sparse Autoencoders (SAEs) provide potentials for uncovering structured, human-interpretable representations in Large Language Models (LLMs), making them a crucial tool for transpa…
Multi-OphthaLingua: A Multilingual Benchmark for Assessing and Debiasing LLM Ophthalmological QA in LMICs
David Restrepo, Chenwei Wu, Zhengxu Tang +14
Current ophthalmology clinical workflows are plagued by over-referrals, long waits, and complex and heterogeneous medical records. Large language models (LLMs) present a promising…
The use of large language models to enhance cancer clinical trial educational materials
Mingye Gao, Aman Varshney, Shan Chen +15
Cancer clinical trials often face challenges in recruitment and engagement due to a lack of participant-facing informational and educational resources. This study investigated the…
WorldMedQA-V: a multilingual, multimodal medical examination dataset for multimodal language models evaluation
João Matos, Shan Chen, Siena Placino +13
Multimodal/vision language models (VLMs) are increasingly being deployed in healthcare settings worldwide, necessitating robust benchmarks to ensure their safety, efficacy, and fai…
Wait, but Tylenol is Acetaminophen... Investigating and Improving Language Models' Ability to Resist Requests for Misinformation
Shan Chen, Mingye Gao, Kuleen Sasse +6
Background: Large language models (LLMs) are trained to follow directions, but this introduces a vulnerability to blindly comply with user requests even if they generate wrong info…
Simulated patient systems powered by large language model-based AI agents offer potential for transforming medical education
Huizi Yu, Jiayan Zhou, Lingyao Li +22
Background: Simulated patient systems are important in medical education and research, providing safe, integrative training environments and supporting clinical decision making. Ad…
EHRmonize: A Framework for Medical Concept Abstraction from Electronic Health Records using Large Language Models
João Matos, Jack Gallifant, Jian Pei +1
Electronic health records (EHRs) contain vast amounts of complex data, but harmonizing and processing this information remains a challenging and costly task requiring significant c…
Cross-Care: Assessing the Healthcare Implications of Pre-training Data on Language Model Bias
Shan Chen, Jack Gallifant, Mingye Gao +12
Large language models (LLMs) are increasingly essential in processing natural languages, yet their application is frequently compromised by biases and inaccuracies originating in t…
Language Models are Surprisingly Fragile to Drug Names in Biomedical Benchmarks
Jack Gallifant, Shan Chen, Pedro Moreira +7
Medical knowledge is context-dependent and requires consistent reasoning across various natural language expressions of semantically equivalent phrases. This is particularly crucia…
ACES: Automatic Cohort Extraction System for Event-Stream Datasets
Justin Xu, Jack Gallifant, Alistair E. W. Johnson +1
Reproducibility remains a significant challenge in machine learning (ML) for healthcare. Datasets, model pipelines, and even task or cohort definitions are often private in this fi…
Analyzing Diversity in Healthcare LLM Research: A Scientometric Perspective
David Restrepo, Chenwei Wu, Constanza Vásquez-Venegas +5
The deployment of large language models (LLMs) in healthcare has demonstrated substantial potential for enhancing clinical decision-making, administrative efficiency, and patient o…
Seeds of Stereotypes: A Large-Scale Textual Analysis of Race and Gender Associations with Diseases in Online Sources
Lasse Hyldig Hansen, Nikolaj Andersen, Jack Gallifant +7
Background Advancements in Large Language Models (LLMs) hold transformative potential in healthcare, however, recent work has raised concern about the tendency of these models to p…
Improving Clinical NLP Performance through Language Model-Generated Synthetic Clinical Data
Shan Chen, Jack Gallifant, Marco Guevara +5
Generative models have been showing potential for producing data in mass. This study explores the enhancement of clinical natural language processing performance by utilizing synth…
A Closer Look at AUROC and AUPRC under Class Imbalance
Matthew B. A. McDermott, Haoran Zhang, Lasse Hyldig Hansen +2
In machine learning (ML), a widespread claim is that the area under the precision-recall curve (AUPRC) is a superior metric for model comparison to the area under the receiver oper…