Towards Demystifying Membership Inference Attacks

Abstract

Membership inference attacks seek to infer membership of individual training instances of a model to which an adversary has black-box access through a machine learning-as-a-service API. In providing an in-depth characterization of membership privacy risks against machine learning models, this paper presents a comprehensive study towards demystifying membership inference attacks from two complimentary perspectives. First, we provide a generalized formulation of the development of a black-box membership inference attack model. Second, we characterize the importance of model choice on model vulnerability through a systematic evaluation of a variety of machine learning models and model combinations using multiple datasets. Through formal analysis and empirical evidence from extensive experimentation, we characterize under what conditions a model may be vulnerable to such black-box membership inference attacks. We show that membership inference vulnerability is data-driven and corresponding attack models are largely transferable. Though different model types display different vulnerabilities to membership inference, so do different datasets. Our empirical results additionally show that (1) using the type of target model under attack within the attack model may not increase attack effectiveness and (2) collaborative learning exposes vulnerabilities to membership inference risks when the adversary is a participant. We also discuss countermeasure and mitigation strategies.

References in corpus (11)

• Scikit-learn: Machine Learning in Python
• Efficient Estimation of Word Representations in Vector Space
• Transferability in Machine Learning: from Phenomena to Black-Box Attacks using Adversarial Samples
• The Secret Sharer: Evaluating and Testing Unintended Memorization in Neural Networks
• Adversarial Examples: Attacks and Defenses for Deep Learning
• Understanding Membership Inferences on Well-Generalized Learning Models
• Towards Measuring Membership Privacy
• Are Accuracy and Robustness Correlated?
• Machine Learning with Membership Privacy using Adversarial Regularization
• LOGAN: Membership Inference Attacks Against Generative Models
• Privacy Risk in Machine Learning: Analyzing the Connection to Overfitting

Cited by in corpus (25)

• A Survey on Large Language Model (LLM) Security and Privacy: The Good, the Bad, and the Ugly
• Privacy in Deep Learning: A Survey
• A Framework for Evaluating Gradient Leakage Attacks in Federated Learning
• Label-Only Membership Inference Attacks
• Data Poisoning Attacks Against Federated Learning Systems
• Modelling and Quantifying Membership Information Leakage in Machine Learning
• Training Data Leakage Analysis in Language Models
• An Overview of Privacy in Machine Learning
• Auditing Data Provenance in Text-Generation Models
• Membership Leakage in Label-Only Exposures
• Membership Inference on Word Embedding and Beyond
• LOGAN: Membership Inference Attacks Against Generative Models
• Towards Security Threats of Deep Learning Systems: A Survey
• Privacy-Preserving Gradient Boosting Decision Trees
• MACE: A Flexible Framework for Membership Privacy Estimation in Generative Models
• Privacy Attacks on Network Embeddings
• NeuGuard: Lightweight Neuron-Guided Defense against Membership Inference Attacks
• PAC learning with stable and private predictions
• Data and Model Dependencies of Membership Inference Attack
• Source Inference Attacks in Federated Learning
• The Connection between Out-of-Distribution Generalization and Privacy of ML Models
• Vulnerability of Face Recognition Systems Against Composite Face Reconstruction Attack
• An Efficient DP-SGD Mechanism for Large Scale NLP Models
• An Extension of Fano's Inequality for Characterizing Model Susceptibility to Membership Inference Attacks
• PrivateMail: Supervised Manifold Learning of Deep Features With Differential Privacy for Image Retrieval