Two Directions for Clinical Data Generation with Large Language Models: Data-to-Label and Label-to-Data

Abstract

Large language models (LLMs) can generate natural language texts for various domains and tasks, but their potential for clinical text mining, a domain with scarce, sensitive, and imbalanced medical data, is underexplored. We investigate whether LLMs can augment clinical data for detecting Alzheimer's Disease (AD)-related signs and symptoms from electronic health records (EHRs), a challenging task that requires high expertise. We create a novel pragmatic taxonomy for AD sign and symptom progression based on expert knowledge, which guides LLMs to generate synthetic data following two different directions: "data-to-label", which labels sentences from a public EHR collection with AD-related signs and symptoms; and "label-to-data", which generates sentences with AD-related signs and symptoms based on the label definition. We train a system to detect AD-related signs and symptoms from EHRs, using three datasets: (1) a gold dataset annotated by human experts on longitudinal EHRs of AD patients; (2) a silver dataset created by the data-to-label method; and (3) a bronze dataset created by the label-to-data method. We find that using the silver and bronze datasets improves the system performance, outperforming the system using only the gold dataset. This shows that LLMs can generate synthetic clinical data for a complex task by incorporating expert knowledge, and our label-to-data method can produce datasets that are free of sensitive information, while maintaining acceptable quality.

Appear in EMNLP2023 Findings

References in corpus (7)

• LLaMA: Open and Efficient Foundation Language Models
• Publicly Available Clinical BERT Embeddings
• LaMDA: Language Models for Dialog Applications
• Capabilities of GPT-4 on Medical Challenge Problems
• Towards Expert-Level Medical Question Answering with Large Language Models
• Understanding the Capabilities, Limitations, and Societal Impact of Large Language Models
• Does Synthetic Data Generation of LLMs Help Clinical Text Mining?

Cited by in corpus (4)

• Large Language Models for Healthcare Text Classification: A Systematic Review
• Explainable cognitive decline detection in free dialogues with a Machine Learning approach based on pre-trained Large Language Models
• A Scoping Review of Synthetic Data Generation by Language Models in Biomedical Research and Application: Data Utility and Quality Perspectives
• Enhancing LLMs for Identifying and Prioritizing Important Medical Jargons from Electronic Health Record Notes Utilizing Data Augmentation: A Comparative Study