Medical Coding with Biomedical Transformer Ensembles and Zero/Few-shot Learning
arXiv:2206.02662 · doi:10.18653/v1/2022.naacl-industry.21
Abstract
Medical coding (MC) is an essential pre-requisite for reliable data retrieval and reporting. Given a free-text reported term (RT) such as "pain of right thigh to the knee", the task is to identify the matching lowest-level term (LLT) - in this case "unilateral leg pain" - from a very large and continuously growing repository of standardized medical terms. However, automating this task is challenging due to a large number of LLT codes (as of writing over 80,000), limited availability of training data for long tail/emerging classes, and the general high accuracy demands of the medical domain. With this paper, we introduce the MC task, discuss its challenges, and present a novel approach called xTARS that combines traditional BERT-based classification with a recent zero/few-shot learning approach (TARS). We present extensive experiments that show that our combined approach outperforms strong baselines, especially in the few-shot regime. The approach is developed and deployed at Bayer, live since November 2021. As we believe our approach potentially promising beyond MC, and to ensure reproducibility, we release the code to the research community.
arXiv commentNAACL-HLT 2022 Industry Track
References in corpus (7)
- BioBERT: a pre-trained biomedical language representation model for biomedical text mining
- Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networks
- Dropout as a Bayesian Approximation: Representing Model Uncertainty in Deep Learning
- Domain-Specific Language Model Pretraining for Biomedical Natural Language Processing
- Underspecification Presents Challenges for Credibility in Modern Machine Learning
- Towards Automated ICD Coding Using Deep Learning
- Read, Attend, and Code: Pushing the Limits of Medical Codes Prediction from Clinical Notes by Machines