An Empirical Evaluation of Deep Learning for ICD-9 Code Assignment using MIMIC-III Clinical Notes

Abstract

Background and Objective: Code assignment is of paramount importance in many levels in modern hospitals, from ensuring accurate billing process to creating a valid record of patient care history. However, the coding process is tedious and subjective, and it requires medical coders with extensive training. This study aims to evaluate the performance of deep-learning-based systems to automatically map clinical notes to ICD-9 medical codes. Methods: The evaluations of this research are focused on end-to-end learning methods without manually defined rules. Traditional machine learning algorithms, as well as state-of-the-art deep learning methods such as Recurrent Neural Networks and Convolution Neural Networks, were applied to the Medical Information Mart for Intensive Care (MIMIC-III) dataset. An extensive number of experiments was applied to different settings of the tested algorithm. Results: Findings showed that the deep learning-based methods outperformed other conventional machine learning methods. From our assessment, the best models could predict the top 10 ICD-9 codes with 0.6957 F1 and 0.8967 accuracy and could estimate the top 10 ICD-9 categories with 0.7233 F1 and 0.8588 accuracy. Our implementation also outperformed existing work under certain evaluation metrics. Conclusion: A set of standard metrics was utilized in assessing the performance of ICD-9 code assignment on MIMIC-III dataset. All the developed evaluation tools and resources are available online, which can be used as a baseline for further research.

References in corpus (7)

• Distributed Representations of Words and Phrases and their Compositionality
• Sequence to Sequence Learning with Neural Networks
• Natural Language Processing (almost) from Scratch
• Distributed Representations of Sentences and Documents
• An Empirical Evaluation of Generic Convolutional and Recurrent Networks for Sequence Modeling
• Deep EHR: A Survey of Recent Advances in Deep Learning Techniques for Electronic Health Record (EHR) Analysis
• Learning to Diagnose with LSTM Recurrent Neural Networks

Cited by in corpus (14)

• Multi-Label Clinical Time-Series Generation via Conditional GAN
• Benchmark datasets driving artificial intelligence development fail to capture the needs of medical professionals
• Natural language processing of MIMIC-III clinical notes for identifying diagnosis and procedures with neural networks
• Experimental Evaluation and Development of a Silver-Standard for the MIMIC-III Clinical Coding Dataset
• Read, Attend, and Code: Pushing the Limits of Medical Codes Prediction from Clinical Notes by Machines
• A Systematic Literature Review of Automated ICD Coding and Classification Systems using Discharge Summaries
• Multi-label natural language processing to identify diagnosis and procedure codes from MIMIC-III inpatient notes
• Predicting Multiple ICD-10 Codes from Brazilian-Portuguese Clinical Notes
• Few-Shot Electronic Health Record Coding through Graph Contrastive Learning
• From Extreme Multi-label to Multi-class: A Hierarchical Approach for Automated ICD-10 Coding Using Phrase-level Attention
• Interpretable ICD Code Embeddings with Self- and Mutual-Attention Mechanisms
• Cross-lingual Data Transformation and Combination for Text Classification
• Graph-Driven Generative Models for Heterogeneous Multi-Task Learning
• Automatic Documentation of ICD Codes with Far-Field Speech Recognition