20 citations · 61 across the 11 of their papers we have counts for
11 papers
HealthAdminBench: Evaluating Computer-Use Agents on Healthcare Administration Tasks
Suhana Bedi, Ryan Welch, Ethan Steinberg +12
Healthcare administration accounts for over $1 trillion in annual spending, making it a promising target for LLM-based computer-use agents (CUAs). While clinical applications of LL…
MedHELM: Holistic Evaluation of Large Language Models for Medical Tasks
Suhana Bedi, Hejie Cui, Miguel Fuentes +78
While large language models (LLMs) achieve near-perfect scores on medical licensing exams, these evaluations inadequately reflect the complexity and diversity of real-world clinica…
Context Clues: Evaluating Long Context Models for Clinical Prediction Tasks on EHRs
Michael Wornow, Suhana Bedi, Miguel Angel Fuentes Hernandez +5
Foundation Models (FMs) trained on Electronic Health Records (EHRs) have achieved state-of-the-art results on numerous clinical prediction tasks. However, most existing EHR FMs hav…
meds_reader: A fast and efficient EHR processing library
Ethan Steinberg, Michael Wornow, Suhana Bedi +3
The growing demand for machine learning in healthcare requires processing increasingly large electronic health record (EHR) datasets, but existing pipelines are not computationally…
WONDERBREAD: A Benchmark for Evaluating Multimodal Foundation Models on Business Process Management Tasks
Michael Wornow, Avanika Narayan, Ben Viggiano +15
Existing ML benchmarks lack the depth and diversity of annotations needed for evaluating models on business process management (BPM) tasks. BPM is the practice of documenting, meas…
Automating the Enterprise with Foundation Models
Michael Wornow, Avanika Narayan, Krista Opsahl-Ong +3
Automating enterprise workflows could unlock $4 trillion/year in productivity gains. Despite being of interest to the data management community for decades, the ultimate vision of…
Recent Advances, Applications, and Open Challenges in Machine Learning for Health: Reflections from Research Roundtables at ML4H 2023 Symposium
Hyewon Jeong, Sarah Jabbour, Yuzhe Yang +40
The third ML4H symposium was held in person on December 10, 2023, in New Orleans, Louisiana, USA. The symposium included research roundtable sessions to foster discussions between…
Standing on FURM ground -- A framework for evaluating Fair, Useful, and Reliable AI Models in healthcare systems
Alison Callahan, Duncan McElfresh, Juan M. Banda +21
The impact of using artificial intelligence (AI) to guide patient care or operational processes is an interplay of the AI model's output, the decision-making protocol based on that…
Zero-Shot Clinical Trial Patient Matching with LLMs
Michael Wornow, Alejandro Lozano, Dev Dash +3
Matching patients to clinical trials is a key unsolved challenge in bringing new drugs to market. Today, identifying patients who meet a trial's eligibility criteria is highly manu…
EHRSHOT: An EHR Benchmark for Few-Shot Evaluation of Foundation Models
Michael Wornow, Rahul Thapa, Ethan Steinberg +2
While the general machine learning (ML) community has benefited from public datasets, tasks, and models, the progress of ML in healthcare has been hampered by a lack of such shared…
The Shaky Foundations of Clinical Foundation Models: A Survey of Large Language Models and Foundation Models for EMRs
Michael Wornow, Yizhe Xu, Rahul Thapa +6
The successes of foundation models such as ChatGPT and AlphaFold have spurred significant interest in building similar models for electronic medical records (EMRs) to improve patie…