Profiling Hallucinations in Frontier LLMs for Entity Linking to Medical Ontologies
arXiv:2611.18595 · doi:10.63317/4zi4vcu7vz4v
Abstract
The integration of Large Language Models (LLMs) into healthcare promises to revolutionize clinical documentation and interoperability, yet reliability remains a concern. This study presents a comprehensive analysis of hallucinations by frontier LLMs tasked with mapping clinical text to SNOMED CT. Through rigorous experimentation, we identify a critical reliability gap: LLMs hallucinate medical codes at a rate that currently renders them unsuitable for autonomous clinical coding applications. Paradoxically, constraining models to use ground-truth mention spans exacerbates, rather than mitigates, these hallucinations. We further contribute a taxonomy of hallucination types--including deprecated codes and cross-ontology errors--and demonstrate that general-purpose LLMs significantly underperform compared to specialized zero-shot entity linking approaches. These findings underscore the need for robust verification mechanisms before clinical deployment.
arXiv commentThe 8th Workshop on Clinical Natural Language Processing at LREC 2026
References in corpus (5)
- A Survey on Hallucination in Large Language Models: Principles, Taxonomy, Challenges, and Open Questions
- HealthBench: Evaluating Large Language Models Towards Improved Human Health
- Large language models are good medical coders, if provided with tools
- Generalist Large Language Models Outperform Clinical Tools on Medical Benchmarks
- Can LLMs Refuse Questions They Do Not Know? Measuring Knowledge-Aware Refusal in Factual Tasks