A Multifaceted Benchmarking of Synthetic Electronic Health Record Generation Models

Abstract

Synthetic health data have the potential to mitigate privacy concerns when sharing data to support biomedical research and the development of innovative healthcare applications. Modern approaches for data generation based on machine learning, generative adversarial networks (GAN) methods in particular, continue to evolve and demonstrate remarkable potential. Yet there is a lack of a systematic assessment framework to benchmark methods as they emerge and determine which methods are most appropriate for which use cases. In this work, we introduce a generalizable benchmarking framework to appraise key characteristics of synthetic health data with respect to utility and privacy metrics. We apply the framework to evaluate synthetic data generation methods for electronic health records (EHRs) data from two large academic medical centers with respect to several use cases. The results illustrate that there is a utility-privacy tradeoff for sharing synthetic EHR data. The results further indicate that no method is unequivocally the best on all criteria in each use case, which makes it evident why synthetic data generation methods need to be assessed in context.

References in corpus (1)

• CovidGAN: Data Augmentation Using Auxiliary Classifier GAN for Improved Covid-19 Detection

Cited by in corpus (7)

• Can I trust my fake data -- A comprehensive quality assessment framework for synthetic tabular data in healthcare
• Synthesize High-dimensional Longitudinal Electronic Health Records via Hierarchical Autoregressive Language Model
• SynthEval: A Framework for Detailed Utility and Privacy Evaluation of Tabular Synthetic Data
• A Consensus Privacy Metrics Framework for Synthetic Data
• Generating Synthetic Electronic Health Record Data: a Methodological Scoping Review with Benchmarking on Phenotype Data and Open-Source Software
• Methods for generating and evaluating synthetic longitudinal patient data: a systematic review
• Evaluation metrics for temporal preservation in synthetic longitudinal patient data