Clinical information extraction for Low-resource languages with Few-shot learning using Pre-trained language models and Prompting

Abstract

Automatic extraction of medical information from clinical documents poses several challenges: high costs of required clinical expertise, limited interpretability of model predictions, restricted computational resources and privacy regulations. Recent advances in domain-adaptation and prompting methods showed promising results with minimal training data using lightweight masked language models, which are suited for well-established interpretability methods. We are first to present a systematic evaluation of these methods in a low-resource setting, by performing multi-class section classification on German doctor's letters. We conduct extensive class-wise evaluations supported by Shapley values, to validate the quality of our small training data set and to ensure the interpretability of model predictions. We demonstrate that a lightweight, domain-adapted pretrained model, prompted with just 20 shots, outperforms a traditional classification model by 30.5% accuracy. Our results serve as a process-oriented guideline for clinical information extraction projects working with low-resource.

Paper accepted for publication in the journal: Natural Language Engineering (Cambridge Core)

References in corpus (4)

• A Comparative Study of Pretrained Language Models for Long Clinical Text
• Evaluating Large Language Models: A Comprehensive Survey
• HealthPrompt: A Zero-shot Learning Paradigm for Clinical Natural Language Processing
• Comprehensive Study on German Language Models for Clinical and Biomedical Text Understanding