6 papers · 1 filter
What's in a Name? Morphological Shortcuts by LLMs in Pharmacology
Kaijie Mo, Thomas Yang, Chantal Shaib +6
The morphological form of a word can often give cues to its meaning, but purely relying on these mappings can lead to overgeneralization in high-stakes domains. In the medical doma…
Decide less, communicate more: On the construct validity of end-to-end fact-checking in medicine
Sebastian Joseph, Lily Chen, Barry Wei +6
Technological progress has led to concrete advancements in tasks that were regarded as challenging, such as automatic fact-checking. Interest in adopting these systems for public h…
Faithfulness vs. Safety: Evaluating LLM Behavior Under Counterfactual Medical Evidence
Kaijie Mo, Siddhartha Venkatayogi, Chantal Shaib +4
In high-stakes domains like medicine, it may be generally desirable for models to faithfully adhere to the context provided. But what happens if the context does not align with mod…
This Treatment Works, Right? Evaluating LLM Sensitivity to Patient Question Framing in Medical QA
Hye Sun Yun, Geetika Kapoor, Michael Mackert +4
Patients are increasingly turning to large language models (LLMs) with medical questions that are complex and difficult to articulate clearly. However, LLMs are sensitive to prompt…
FactPICO: Factuality Evaluation for Plain Language Summarization of Medical Evidence
Sebastian Antony Joseph, Lily Chen, Jan Trienes +5
Plain language summarization with LLMs can be useful for improving textual accessibility of technical content. But how factual are these summaries in a high-stakes domain like medi…
InfoLossQA: Characterizing and Recovering Information Loss in Text Simplification
Jan Trienes, Sebastian Joseph, Jörg Schlötterer +5
Text simplification aims to make technical texts more accessible to laypeople but often results in deletion of information and vagueness. This work proposes InfoLossQA, a framework…