11 citations · 37 across the 12 of their papers we have counts for
13 papers
A Living Benchmark for Information Retrieval from Electronic Health Records
Jordan L. Cahoon, Chloe O. Stanwyck, Sulaiman Somani +23
Large language model (LLM)-based clinical assistants are increasingly being integrated into electronic health record (EHR) systems, transforming how clinicians retrieve and synthes…
"Are you an AI?" Analyzing Client Suspicion of AI Use in Crisis Counseling
Shreya Shah, Akshay Swaminathan, Meghana Simhadri +14
As artificial intelligence (AI) tools get increasingly deployed for mental healthcare, public trust in these systems remains uncertain. It is unclear how clients perceive AI involv…
Structured Prompts Improve Evaluation of Language Models
Asad Aali, Muhammad Ahmed Mohsin, Vasiliki Bikia +15
As language models (LMs) are increasingly adopted across domains, high-quality benchmarking frameworks are essential for guiding deployment decisions. However, frameworks such as H…
MedFactEval and MedAgentBrief: A Framework and Workflow for Generating and Evaluating Factual Clinical Summaries
François Grolleau, Emily Alsentzer, Timothy Keyes +17
Evaluating factual accuracy in Large Language Model (LLM)-generated clinical text is a critical barrier to adoption, as expert review is unscalable for the continuous quality assur…
MedHELM: Holistic Evaluation of Large Language Models for Medical Tasks
Suhana Bedi, Hejie Cui, Miguel Fuentes +78
While large language models (LLMs) achieve near-perfect scores on medical licensing exams, these evaluations inadequately reflect the complexity and diversity of real-world clinica…
VeriFact: Verifying Facts in LLM-Generated Clinical Text with Electronic Health Records
Philip Chung, Akshay Swaminathan, Alex J. Goodell +26
Methods to ensure factual accuracy of text generated by large language models (LLM) in clinical medicine are lacking. VeriFact is an artificial intelligence system that combines re…
Distilling Large Language Models for Efficient Clinical Information Extraction
Karthik S. Vedula, Annika Gupta, Akshay Swaminathan +3
Large language models (LLMs) excel at clinical information extraction but their computational demands limit practical deployment. Knowledge distillation--the process of transferrin…
FactEHR: A Dataset for Evaluating Factuality in Clinical Notes Using LLMs
Monica Munnangi, Akshay Swaminathan, Jason Alan Fries +8
Verifying and attributing factual claims is essential for the safe and effective use of large language models (LLMs) in healthcare. A core component of factuality evaluation is fac…
Answering real-world clinical questions using large language model based systems
Yen Sia Low, Michael L. Jackson, Rebecca J. Hyde +24
Evidence to guide healthcare decisions is often limited by a lack of relevant and trustworthy literature as well as difficulty in contextualizing existing research for a specific p…
WONDERBREAD: A Benchmark for Evaluating Multimodal Foundation Models on Business Process Management Tasks
Michael Wornow, Avanika Narayan, Ben Viggiano +15
Existing ML benchmarks lack the depth and diversity of annotations needed for evaluating models on business process management (BPM) tasks. BPM is the practice of documenting, meas…
Automating the Enterprise with Foundation Models
Michael Wornow, Avanika Narayan, Krista Opsahl-Ong +3
Automating enterprise workflows could unlock $4 trillion/year in productivity gains. Despite being of interest to the data management community for decades, the ultimate vision of…
Zero-Shot Clinical Trial Patient Matching with LLMs
Michael Wornow, Alejandro Lozano, Dev Dash +3
Matching patients to clinical trials is a key unsolved challenge in bringing new drugs to market. Today, identifying patients who meet a trial's eligibility criteria is highly manu…
Clinfo.ai: An Open-Source Retrieval-Augmented Large Language Model System for Answering Medical Questions using Scientific Literature
Alejandro Lozano, Scott L Fleming, Chia-Chun Chiang +1
The quickly-expanding nature of published medical literature makes it challenging for clinicians and researchers to keep up with and summarize recent, relevant findings in a timely…