10 citations · 12 across the 7 of their papers we have counts for
8 papers
A primer on evaluation methods for large language models in healthcare
Suzannah E McKinney, Phuc Vu, Samuel A Justice +12
Large language models (LLMs) have a growing range of applications in medicine, and their evaluation is critical for ensuring they provide benefit and not harm. This evaluation can…
Development, Evaluation, and Deployment of a Multi-Agent System for Thoracic Tumor Board
Tim Ellis-Caleo, Timothy Keyes, Nerissa Ambers +5
Tumor boards are multidisciplinary conferences dedicated to producing actionable patient care recommendations with live review of primary radiology and pathology data. Succinct pat…
Multi-Method Validation of Large Language Model Medical Translation Across High- and Low-Resource Languages
Chukwuebuka Anyaegbuna, Eduardo Juan Perez Guerrero, Jerry Liu +6
Language barriers affect 27.3 million U.S. residents with non-English language preference, yet professional medical translation remains costly and often unavailable. We evaluated f…
Deployment and Evaluation of an EHR-integrated, Large Language Model-Powered Tool to Triage Surgical Patients
Jane Wang, Timothy Keyes, April S Liang +10
Surgical co-management (SCM) is an evidence-based model in which hospitalists jointly manage medically complex perioperative patients alongside surgical teams. Despite its clinical…
Adoption and Use of LLMs at an Academic Medical Center
Nigam H. Shah, Nerissa Ambers, Abby Pandya +55
While large language models (LLMs) can support clinical documentation needs, standalone tools struggle with "workflow friction" from manual data entry. We developed ChatEHR, a syst…
Monitoring Deployed AI Systems in Health Care
Timothy Keyes, Alison Callahan, Abby S. Pandya +18
Post-deployment monitoring of artificial intelligence (AI) systems in health care is essential to ensure their safety, quality, and sustained benefit-and to support governance deci…
MedFactEval and MedAgentBrief: A Framework and Workflow for Generating and Evaluating Factual Clinical Summaries
François Grolleau, Emily Alsentzer, Timothy Keyes +17
Evaluating factual accuracy in Large Language Model (LLM)-generated clinical text is a critical barrier to adoption, as expert review is unscalable for the continuous quality assur…
MedHELM: Holistic Evaluation of Large Language Models for Medical Tasks
Suhana Bedi, Hejie Cui, Miguel Fuentes +78
While large language models (LLMs) achieve near-perfect scores on medical licensing exams, these evaluations inadequately reflect the complexity and diversity of real-world clinica…