5 papers
MechaTerp-TRACE: A Novel Approach for Component Ablation Analysis in Language Models
Brandon Colelough, Davis Bartels, Madeline Bittner +1
Interpretability research on large language models has produced accounts of factual recall in feed-forward layers and of token relationships in self-attention, but little work offe…
A Dataset and Resources for Identifying Patient Health Literacy Information from Clinical Notes
Madeline Bittner, Dina Demner-Fushman, Yasmeen Shabazz +6
Health literacy is a critical determinant of patient outcomes, yet current screening tools are not always feasible and differ considerably in the number of items, question format,…
Quantifying Hallucinations in Language Language Models on Medical Textbooks
Brandon C. Colelough, Davis Bartels, Dina Demner-Fushman
Hallucinations, the tendency for large language models to provide responses with factually incorrect and unsupported claims, is a serious problem within natural language processing…
BioACE: An Automated Framework for Biomedical Answer and Citation Evaluations
Deepak Gupta, Davis Bartels, Dina Demner-Fushman
With the increasing use of large language models (LLMs) for generating answers to biomedical questions, it is crucial to evaluate the quality of the generated answers and the refer…
Overview of the ClinIQLink 2025 Shared Task on Medical Question-Answering
Brandon Colelough, Davis Bartels, Dina Demner-Fushman
In this paper, we present an overview of ClinIQLink, a shared task, collocated with the 24th BioNLP workshop at ACL 2025, designed to stress-test large language models (LLMs) on me…