Differentially Private Synthetic Data: Applied Evaluations and Enhancements
arXiv:2011.05537
Abstract
Machine learning practitioners frequently seek to leverage the most informative available data, without violating the data owner's privacy, when building predictive models. Differentially private data synthesis protects personal details from exposure, and allows for the training of differentially private machine learning models on privately generated datasets. But how can we effectively assess the efficacy of differentially private synthetic data? In this paper, we survey four differentially private generative adversarial networks for data synthesis. We evaluate each of them at scale on five standard tabular datasets, and in two applied industry scenarios. We benchmark with novel metrics from recent literature and other standard machine learning tools. Our results suggest some synthesizers are more applicable for different privacy budgets, and we further demonstrate complicating domain-based tradeoffs in selecting an approach. We offer experimental learning on applied machine learning scenarios with private internal data to researchers and practioners alike. In addition, we propose QUAIL, an ensemble-based modeling approach to generating synthetic data. We examine QUAIL's tradeoffs, and note circumstances in which it outperforms baseline differentially private supervised learning models under the same budget constraint.
Under Review
References in corpus (18)
- Scikit-learn: Machine Learning in Python
- Deep Learning with Differential Privacy
- Data Synthesis based on Generative Adversarial Networks
- Differentially Private Generative Adversarial Network
- GAN-Leaks: A Taxonomy of Membership Inference Attacks against Generative Models
- A simple and practical algorithm for differentially private data release
- Differential Privacy and Machine Learning: a Survey and Review
- Privacy Amplification by Iteration
- Modeling Tabular data using Conditional GAN
- DP-GAN: Diversity-Promoting Generative Adversarial Network for Generating Informative and Diversified Text
- AutoGAN-based Dimension Reduction for Privacy Preservation
- Differentially Private Releasing via Deep Generative Model (Technical Report)
- Private Approximations of the 2nd-Moment Matrix Using Existing Techniques in Linear Regression
- Really Useful Synthetic Data -- A Framework to Evaluate the Quality of Differentially Private Synthetic Data
- Measuring the quality of Synthetic data for use in competitions
- One-sided Differential Privacy
- privGAN: Protecting GANs from membership inference attacks at low cost
- On the Differential Privacy of Bayesian Inference