Synthesize High-dimensional Longitudinal Electronic Health Records via Hierarchical Autoregressive Language Model

Abstract

Synthetic electronic health records (EHRs) that are both realistic and preserve privacy can serve as an alternative to real EHRs for machine learning (ML) modeling and statistical analysis. However, generating high-fidelity and granular electronic health record (EHR) data in its original, highly-dimensional form poses challenges for existing methods due to the complexities inherent in high-dimensional data. In this paper, we propose Hierarchical Autoregressive Language mOdel (HALO) for generating longitudinal high-dimensional EHR, which preserve the statistical properties of real EHR and can be used to train accurate ML models without privacy concerns. Our HALO method, designed as a hierarchical autoregressive model, generates a probability density function of medical codes, clinical visits, and patient records, allowing for the generation of realistic EHR data in its original, unaggregated form without the need for variable selection or aggregation. Additionally, our model also produces high-quality continuous variables in a longitudinal and probabilistic manner. We conducted extensive experiments and demonstrate that HALO can generate high-fidelity EHR data with high-dimensional disease code probabilities (d > 10,000), disease co-occurrence probabilities within visits (d > 1,000,000), and conditional probabilities across consecutive visits (d > 5,000,000) and achieve above 0.9 R2 correlation in comparison to real EHR data. This performance then enables downstream ML models trained on its synthetic data to achieve comparable accuracy to models trained on real data (0.938 AUROC with HALO data vs. 0.943 with real data). Finally, using a combination of real and synthetic data enhances the accuracy of ML models beyond that achieved by using only real EHR data.

References in corpus (5)

• PyTorch: An Imperative Style, High-Performance Deep Learning Library
• RETAIN: An Interpretable Predictive Model for Healthcare using Reverse Time Attention Mechanism
• A Multifaceted Benchmarking of Synthetic Electronic Health Record Generation Models
• REST: Robust and Efficient Neural Networks for Sleep Monitoring in the Wild
• EVA: Generating Longitudinal Electronic Health Records Using Conditional Variational Autoencoders

Cited by in corpus (6)

• Generative AI for Synthetic Data Across Multiple Medical Modalities: A Systematic Review of Recent Developments and Challenges
• Can I trust my fake data -- A comprehensive quality assessment framework for synthetic tabular data in healthcare
• A Scoping Review of Synthetic Data Generation by Language Models in Biomedical Research and Application: Data Utility and Quality Perspectives
• Generating Synthetic Electronic Health Record Data: a Methodological Scoping Review with Benchmarking on Phenotype Data and Open-Source Software
• Methods for generating and evaluating synthetic longitudinal patient data: a systematic review
• Evaluation metrics for temporal preservation in synthetic longitudinal patient data