BEHRT: Transformer for Electronic Health Records

Abstract

Today, despite decades of developments in medicine and the growing interest in precision healthcare, vast majority of diagnoses happen once patients begin to show noticeable signs of illness. Early indication and detection of diseases, however, can provide patients and carers with the chance of early intervention, better disease management, and efficient allocation of healthcare resources. The latest developments in machine learning (more specifically, deep learning) provides a great opportunity to address this unmet need. In this study, we introduce BEHRT: A deep neural sequence transduction model for EHR (electronic health records), capable of multitask prediction and disease trajectory mapping. When trained and evaluated on the data from nearly 1.6 million individuals, BEHRT shows a striking absolute improvement of 8.0-10.8%, in terms of Average Precision Score, compared to the existing state-of-the-art deep EHR models (in terms of average precision, when predicting for the onset of 301 conditions). In addition to its superior prediction power, BEHRT provides a personalised view of disease trajectories through its attention mechanism; its flexible architecture enables it to incorporate multiple heterogeneous concepts (e.g., diagnosis, medication, measurements, and more) to improve the accuracy of its predictions; and its (pre-)training results in disease and patient representations that can help us get a step closer to interpretable predictions.

arXiv commentShishir Rao and Yikuan Li have equally contributed to this work as first authors. Shishir Rao is Corresponding Author for this work

References in corpus (6)

• Practical Bayesian Optimization of Machine Learning Algorithms
• Evaluation: from precision, recall and F-measure to ROC, informedness, markedness and correlation
• Learning Phrase Representations using RNN Encoder-Decoder for Statistical Machine Translation
• Predicting Cardiovascular Risk Factors from Retinal Fundus Photographs using Deep Learning
• Deep EHR: A Survey of Recent Advances in Deep Learning Techniques for Electronic Health Record (EHR) Analysis
• DeepCare: A Deep Dynamic Memory Model for Predictive Medicine

Cited by in corpus (4)

• SYMBA: Symbolic Computation of Squared Amplitudes in High Energy Physics with Machine Learning
• Neural Temporal Point Processes For Modelling Electronic Health Records
• MUFASA: Multimodal Fusion Architecture Search for Electronic Health Records
• Deep Semi-Supervised Embedded Clustering (DSEC) for Stratification of Heart Failure Patients