Auditing evaluation artifacts and data leakage in large language model systems for rare disease diagnosis

Jian Yang, Peilin Liu, Xiaohan Li +4

Augmented large language model (LLM) systems have reported high benchmark accuracies in rare disease diagnosis, yet their real-world clinical utility remains unproven. A recent met…