Knowledge Cross-Distillation for Membership Privacy
arXiv:2111.01363 · doi:10.2478/popets-2022-0050
Abstract
A membership inference attack (MIA) poses privacy risks for the training data of a machine learning model. With an MIA, an attacker guesses if the target data are a member of the training dataset. The state-of-the-art defense against MIAs, distillation for membership privacy (DMP), requires not only private data for protection but a large amount of unlabeled public data. However, in certain privacy-sensitive domains, such as medicine and finance, the availability of public data is not guaranteed. Moreover, a trivial method for generating public data by using generative adversarial networks significantly decreases the model accuracy, as reported by the authors of DMP. To overcome this problem, we propose a novel defense against MIAs that uses knowledge distillation without requiring public data. Our experiments show that the privacy protection and accuracy of our defense are comparable to those of DMP for the benchmark tabular datasets used in MIA research, Purchase100 and Texas100, and our defense has a much better privacy-utility trade-off than those of the existing defenses that also do not use public data for the image dataset CIFAR10.
arXiv commentAccepted by PETS 2022
References in corpus (20)
- Auto-Encoding Variational Bayes
- Distilling the Knowledge in a Neural Network
- Unsupervised Representation Learning with Deep Convolutional Generative Adversarial Networks
- Deep Learning with Differential Privacy
- Intriguing properties of neural networks
- Comprehensive Privacy Analysis of Deep Learning: Passive and Active White-box Inference Attacks against Centralized and Federated Learning
- The Secret Sharer: Evaluating and Testing Unintended Memorization in Neural Networks
- Scalable Private Learning with PATE
- GAN-Leaks: A Taxonomy of Membership Inference Attacks against Generative Models
- Algorithms that Remember: Model Inversion Attacks and Data Protection Law
- Semi-supervised Knowledge Transfer for Deep Learning from Private Training Data
- Privacy Risks of Securing Machine Learning Models against Adversarial Examples
- Understanding Membership Inferences on Well-Generalized Learning Models
- White-box vs Black-box: Bayes Optimal Strategies for Membership Inference
- Stolen Memories: Leveraging Model Memorization for Calibrated White-Box Membership Inference
- Label-Only Membership Inference Attacks
- Cronus: Robust and Heterogeneous Collaborative Learning with Black-Box Knowledge Transfer
- Practical Blind Membership Inference Attack via Differential Comparisons
- Mitigating Membership Inference Attacks by Self-Distillation Through a Novel Ensemble Architecture
- Membership Privacy for Machine Learning Models Through Knowledge Transfer