Bounding Information Leakage in Machine Learning
arXiv:2105.03875 · doi:10.1016/j.neucom.2023.02.058
Abstract
Recently, it has been shown that Machine Learning models can leak sensitive information about their training data. This information leakage is exposed through membership and attribute inference attacks. Although many attack strategies have been proposed, little effort has been made to formalize these problems. We present a novel formalism, generalizing membership and attribute inference attack setups previously studied in the literature and connecting them to memorization and generalization. First, we derive a universal bound on the success rate of inference attacks and connect it to the generalization gap of the target model. Second, we study the question of how much sensitive information is stored by the algorithm about its training set and we derive bounds on the mutual information between the sensitive attributes and model parameters. Experimentally, we illustrate the potential of our approach by applying it to both synthetic data and classification tasks on natural images. Finally, we apply our formalism to different attribute inference strategies, with which an adversary is able to recover the identity of writers in the PenDigits dataset.
arXiv commentPublished in [Elsevier Neurocomputing](https://doi.org/10.1016/j.neucom.2023.02.058)
References in corpus (9)
- Deep Learning with Differential Privacy
- DeepSurv: Personalized Treatment Recommender System Using A Cox Proportional Hazards Deep Neural Network
- An overview of deep learning in medical imaging focusing on MRI
- Comprehensive Privacy Analysis of Deep Learning: Passive and Active White-box Inference Attacks against Centralized and Federated Learning
- GAN-Leaks: A Taxonomy of Membership Inference Attacks against Generative Models
- Privacy Risks of Securing Machine Learning Models against Adversarial Examples
- Learners that Use Little Information
- Not one but many Tradeoffs: Privacy Vs. Utility in Differentially Private Machine Learning
- Membership Model Inversion Attacks for Deep Networks