5 papers · 1 filter
FHIR-AgentBench: Benchmarking LLM Agents for Realistic Interoperable EHR Question Answering
Gyubok Lee, Elea Bach, Eric Yang +5
The recent shift toward the Health Level Seven Fast Healthcare Interoperability Resources (HL7 FHIR) standard opens a new frontier for clinical AI, demanding LLM agents to navigate…
An AI-Based Behavioral Health Safety Filter and Dataset for Identifying Mental Health Crises in Text-Based Conversations
Benjamin W. Nelson, Celeste Wong, Matthew T. Silvestrini +6
Large language models often mishandle psychiatric emergencies, offering harmful or inappropriate advice and enabling destructive behaviors. This study evaluated the Verily behavior…
Sleepless Nights, Sugary Days: Creating Synthetic Users with Health Conditions for Realistic Coaching Agent Interactions
Taedong Yun, Eric Yang, Mustafa Safdari +13
We present an end-to-end framework for generating synthetic users for evaluating interactive agents designed to encourage positive behavior changes, such as in health and lifestyle…
The Geometry of Queries: Query-Based Innovations in Retrieval-Augmented Generation for Healthcare QA
Eric Yang, Jonathan Amar, Jong Ha Lee +2
Deploying Large Language Models (LLMs) for healthcare question answering requires robust methods to ensure accuracy and reliability. This work introduces Query-Based Retrieval Augm…
AI Agents for Conversational Patient Triage: Preliminary Simulation-Based Evaluation with Real-World EHR Data
Sina Rashidian, Nan Li, Jonathan Amar +7
Background: We present a Patient Simulator that leverages real world patient encounters which cover a broad range of conditions and symptoms to provide synthetic test subjects for…