Differentially Private Data Generative Models

Abstract

Deep neural networks (DNNs) have recently been widely adopted in various applications, and such success is largely due to a combination of algorithmic breakthroughs, computation resource improvements, and access to a large amount of data. However, the large-scale data collections required for deep learning often contain sensitive information, therefore raising many privacy concerns. Prior research has shown several successful attacks in inferring sensitive training data information, such as model inversion, membership inference, and generative adversarial networks (GAN) based leakage attacks against collaborative deep learning. In this paper, to enable learning efficiency as well as to generate data with privacy guarantees and high utility, we propose a differentially private autoencoder-based generative model (DP-AuGM) and a differentially private variational autoencoder-based generative model (DP-VaeGM). We evaluate the robustness of two proposed models. We show that DP-AuGM can effectively defend against the model inversion, membership inference, and GAN-based attacks. We also show that DP-VaeGM is robust against the membership inference attack. We conjecture that the key to defend against the model inversion and GAN-based attacks is not due to differential privacy but the perturbation of training data. Finally, we demonstrate that both DP-AuGM and DP-VaeGM can be easily integrated with real-world machine learning applications, such as machine learning as a service and federated learning, which are otherwise threatened by the membership inference attack and the GAN-based attack, respectively.

References in corpus (8)

• Auto-Encoding Variational Bayes
• Unsupervised Representation Learning with Deep Convolutional Generative Adversarial Networks
• Deep Learning with Differential Privacy
• Scalable Private Learning with PATE
• Towards Vision-Based Deep Reinforcement Learning for Robotic Motion Control
• A Framework for Individualizing Predictions of Disease Trajectories by Exploiting Multi-Resolution Structure
• Learning Privately from Multiparty Data
• Performing Co-Membership Attacks Against Deep Generative Models

Cited by in corpus (16)

• Federated Learning for Healthcare Informatics
• Node-Level Membership Inference Attacks Against Graph Neural Networks
• Stealing Links from Graph Neural Networks
• DP-MERF: Differentially Private Mean Embeddings with Random Features for Practical Privacy-Preserving Data Generation
• Kamino: Constraint-Aware Differentially Private Data Synthesis
• Wide Network Learning with Differential Privacy
• Differentially Private Deep Learning with Direct Feedback Alignment
• DPD-InfoGAN: Differentially Private Distributed InfoGAN
• EncoderMI: Membership Inference against Pre-trained Encoders in Contrastive Learning
• Matrix Sketching for Secure Collaborative Machine Learning
• Private Dataset Generation Using Privacy Preserving Collaborative Learning
• TableGAN-MCA: Evaluating Membership Collisions of GAN-Synthesized Tabular Data Releasing
• DP-CGAN: Differentially Private Synthetic Data and Label Generation
• Scalability vs. Utility: Do We Have to Sacrifice One for the Other in Data Importance Quantification?
• Loosely Coupled Federated Learning Over Generative Models
• Synthetic Data Generation for Economists