2 papers
cs.CL2025
MedPI: Evaluating AI Systems in Medical Patient-facing Interactions
Diego Fajardo V., Oleksii Proniakin, Victoria-Elisabeth Gruber +1
We present MedPI, a high-dimensional benchmark for evaluating large language models (LLMs) in patient-clinician conversations. Unlike single-turn question-answer (QA) benchmarks, M…
cs.CL2025
Automatic Replication of LLM Mistakes in Medical Conversations
Oleksii Proniakin, Diego Fajardo, Ruslan Nazarenko +1
Large language models (LLMs) are increasingly evaluated in clinical settings using multi-dimensional rubrics which quantify reasoning quality, safety, and patient-centeredness. Yet…