PEARL: Data Synthesis via Private Embeddings and Adversarial Reconstruction Learning
arXiv:2106.04590
Abstract
We propose a new framework of synthesizing data using deep generative models in a differentially private manner. Within our framework, sensitive data are sanitized with rigorous privacy guarantees in a one-shot fashion, such that training deep generative models is possible without re-using the original data. Hence, no extra privacy costs or model constraints are incurred, in contrast to popular approaches such as Differentially Private Stochastic Gradient Descent (DP-SGD), which, among other issues, causes degradation in privacy guarantees as the training iteration increases. We demonstrate a realization of our framework by making use of the characteristic function and an adversarial re-weighting objective, which are of independent interest as well. Our proposal has theoretical guarantees of performance, and empirical evaluations on multiple datasets show that our approach outperforms other methods at reasonable levels of privacy.
arXiv comment22 pages, 10 figures, accepted to ICLR 2022
References in corpus (16)
- Scikit-learn: Machine Learning in Python
- Auto-Encoding Variational Bayes
- Conditional Generative Adversarial Nets
- Deep Learning with Differential Privacy
- Fashion-MNIST: a Novel Image Dataset for Benchmarking Machine Learning Algorithms
- GANs Trained by a Two Time-Scale Update Rule Converge to a Local Nash Equilibrium
- Renyi Differential Privacy
- MMD GAN: Towards Deeper Understanding of Moment Matching Network
- Differentially Private Generative Adversarial Network
- A simple and practical algorithm for differentially private data release
- The Cramer Distance as a Solution to Biased Wasserstein Gradients
- Subsampled Rényi Differential Privacy and Analytical Moments Accountant
- Fast Two-Sample Testing with Analytic Representations of Probability Measures
- Large sample analysis of the median heuristic
- P3GM: Private High-Dimensional Data Release via Privacy Preserving Phased Generative Model
- Differentially Private Variational Autoencoders with Term-wise Gradient Aggregation