Stolen Memories: Leveraging Model Memorization for Calibrated White-Box Membership Inference

Abstract

Membership inference (MI) attacks exploit the fact that machine learning algorithms sometimes leak information about their training data through the learned model. In this work, we study membership inference in the white-box setting in order to exploit the internals of a model, which have not been effectively utilized by previous work. Leveraging new insights about how overfitting occurs in deep neural networks, we show how a model's idiosyncratic use of features can provide evidence for membership to white-box attackers---even when the model's black-box behavior appears to generalize well---and demonstrate that this attack outperforms prior black-box methods. Taking the position that an effective attack should have the ability to provide confident positive inferences, we find that previous attacks do not often provide a meaningful basis for confidently inferring membership, whereas our attack can be effectively calibrated for high precision. Finally, we examine popular defenses against MI attacks, finding that (1) smaller generalization error is not sufficient to prevent attacks on real models, and (2) while small-$\epsilon$-differential privacy reduces the attack's effectiveness, this often comes at a significant cost to the model's accuracy; and for larger $\epsilon$ that are sometimes used in practice (e.g., $\epsilon =16$), the attack can achieve nearly the same accuracy as on the unprotected model.

arXiv commentappearing in USENIX 2020

Cited by in corpus (28)

• Label-Only Membership Inference Attacks
• Systematic Evaluation of Privacy Risks of Machine Learning Models
• Node-Level Membership Inference Attacks Against Graph Neural Networks
• Defending Model Inversion and Membership Inference Attacks via Prediction Purification
• ML-Doctor: Holistic Risk Assessment of Inference Attacks Against Machine Learning Models
• Stealing Links from Graph Neural Networks
• Modelling and Quantifying Membership Information Leakage in Machine Learning
• Training Data Leakage Analysis in Language Models
• Evaluation of Inference Attack Models for Deep Learning on Medical Data
• Knowledge Cross-Distillation for Membership Privacy
• Membership Leakage in Label-Only Exposures
• Membership Inference on Word Embedding and Beyond
• Enhanced Membership Inference Attacks against Machine Learning Models
• Adversarial Machine Learning for 5G Communications Security
• SoK: Machine Learning Governance
• Synthetic Data -- Anonymisation Groundhog Day
• Membership Inference Attacks Against Recommender Systems
• FaceLeaks: Inference Attacks against Transfer Learning Models via Black-box Queries
• Quantifying and Mitigating Privacy Risks of Contrastive Learning
• On the Importance of Difficulty Calibration in Membership Inference Attacks
• Data and Model Dependencies of Membership Inference Attack
• Teacher Model Fingerprinting Attacks Against Transfer Learning
• Selective Ensembles for Consistent Predictions
• On the Difficulty of Membership Inference Attacks
• Generative Models for Security: Attacks, Defenses, and Opportunities
• Over-the-Air Membership Inference Attacks as Privacy Threats for Deep Learning-based Wireless Signal Classifiers
• Topology-aware Differential Privacy for Decentralized Image Classification
• CAPE: Context-Aware Private Embeddings for Private Language Learning