GS-WGAN: A Gradient-Sanitized Approach for Learning Differentially Private Generators

Abstract

The wide-spread availability of rich data has fueled the growth of machine learning applications in numerous domains. However, growth in domains with highly-sensitive data (e.g., medical) is largely hindered as the private nature of data prohibits it from being shared. To this end, we propose Gradient-sanitized Wasserstein Generative Adversarial Networks (GS-WGAN), which allows releasing a sanitized form of the sensitive data with rigorous privacy guarantees. In contrast to prior work, our approach is able to distort gradient information more precisely, and thereby enabling training deeper models which generate more informative samples. Moreover, our formulation naturally allows for training GANs in both centralized and federated (i.e., decentralized) data scenarios. Through extensive experiments, we find our approach consistently outperforms state-of-the-art approaches across multiple metrics (e.g., sample quality) and datasets.

arXiv commentNeurIPS 2020, 18 pages

References in corpus (16)

• Scikit-learn: Machine Learning in Python
• Deep Learning with Differential Privacy
• Fashion-MNIST: a Novel Image Dataset for Benchmarking Machine Learning Algorithms
• Communication-Efficient Learning of Deep Networks from Decentralized Data
• GANs Trained by a Two Time-Scale Update Rule Converge to a Local Nash Equilibrium
• Large Scale GAN Training for High Fidelity Natural Image Synthesis
• Rényi Divergence and Kullback-Leibler Divergence
• Renyi Differential Privacy
• Learning Differentially Private Recurrent Language Models
• Differentially Private Generative Adversarial Network
• LEAF: A Benchmark for Federated Settings
• Privacy Amplification by Subsampling: Tight Analyses via Couplings and Divergences
• Subsampled Rényi Differential Privacy and Analytical Moments Accountant
• Differentially Private Releasing via Deep Generative Model (Technical Report)
• Graphical-model based estimation and inference for differential privacy
• DP-MERF: Differentially Private Mean Embeddings with Random Features for Practical Privacy-Preserving Data Generation

Cited by in corpus (10)

• GAN-Leaks: A Taxonomy of Membership Inference Attacks against Generative Models
• Large Language Models Can Be Strong Differentially Private Learners
• DTGAN: Differential Private Training for Tabular GANs
• Does Differentially Private Synthetic Data Lead to Synthetic Discoveries?
• Privacy Inference Attacks and Defenses in Cloud-based Deep Neural Network: A Survey
• PATE-AAE: Incorporating Adversarial Autoencoder into Private Aggregation of Teacher Ensembles for Spoken Command Classification
• Differentially Private Data Generation with Missing Data
• Effective and Privacy preserving Tabular Data Synthesizing
• Perceptual Indistinguishability-Net (PI-Net): Facial Image Obfuscation with Manipulable Semantics
• Privacy-Enhanced Database Synthesis for Benchmark Publishing (Technical Report)