Large Language Models are Few-Shot Clinical Information Extractors
arXiv:2205.12689
Abstract
A long-running goal of the clinical NLP community is the extraction of important variables trapped in clinical notes. However, roadblocks have included dataset shift from the general domain and a lack of public clinical corpora and annotations. In this work, we show that large language models, such as InstructGPT, perform well at zero- and few-shot information extraction from clinical text despite not being trained specifically for the clinical domain. Whereas text classification and generation performance have already been studied extensively in such models, here we additionally demonstrate how to leverage them to tackle a diverse set of NLP tasks which require more structured outputs, including span identification, token-level sequence classification, and relation extraction. Further, due to the dearth of available data to evaluate these systems, we introduce new datasets for benchmarking few-shot clinical information extraction based on a manual re-annotation of the CASI dataset for new tasks. On the clinical extraction tasks we studied, the GPT-3 systems significantly outperform existing zero- and few-shot baselines.
Accepted as a long paper to The 2022 Conference on Empirical Methods in Natural Language Processing (EMNLP)
Cited by in corpus (4)
- BioGPT: Generative Pre-trained Transformer for Biomedical Text Generation and Mining
- Retrieving Evidence from EHRs with LLMs: Possibilities and Challenges
- BigBIO: A Framework for Data-Centric Biomedical Natural Language Processing
- Leveraging Large Language Models for Knowledge-free Weak Supervision in Clinical Natural Language Processing