5 papers
STEER: Inference-Time Risk Control via Constrained Quality-Diversity Search
Eric Yang, Jong Ha Lee, Jonathan Amar +2
Large Language Models (LLMs) trained for average correctness often exhibit mode collapse, producing narrow decision behaviors on tasks where multiple responses may be reasonable. T…
FHIR-AgentBench: Benchmarking LLM Agents for Realistic Interoperable EHR Question Answering
Gyubok Lee, Elea Bach, Eric Yang +5
The recent shift toward the Health Level Seven Fast Healthcare Interoperability Resources (HL7 FHIR) standard opens a new frontier for clinical AI, demanding LLM agents to navigate…
Sleepless Nights, Sugary Days: Creating Synthetic Users with Health Conditions for Realistic Coaching Agent Interactions
Taedong Yun, Eric Yang, Mustafa Safdari +13
We present an end-to-end framework for generating synthetic users for evaluating interactive agents designed to encourage positive behavior changes, such as in health and lifestyle…
The Geometry of Queries: Query-Based Innovations in Retrieval-Augmented Generation for Healthcare QA
Eric Yang, Jonathan Amar, Jong Ha Lee +2
Deploying Large Language Models (LLMs) for healthcare question answering requires robust methods to ensure accuracy and reliability. This work introduces Query-Based Retrieval Augm…
AI Agents for Conversational Patient Triage: Preliminary Simulation-Based Evaluation with Real-World EHR Data
Sina Rashidian, Nan Li, Jonathan Amar +7
Background: We present a Patient Simulator that leverages real world patient encounters which cover a broad range of conditions and symptoms to provide synthetic test subjects for…