Membership Inference Attacks and Generalization: A Causal Perspective

Abstract

Membership inference (MI) attacks highlight a privacy weakness in present stochastic training methods for neural networks. It is not well understood, however, why they arise. Are they a natural consequence of imperfect generalization only? Which underlying causes should we address during training to mitigate these attacks? Towards answering such questions, we propose the first approach to explain MI attacks and their connection to generalization based on principled causal reasoning. We offer causal graphs that quantitatively explain the observed MI attack performance achieved for $6$ attack variants. We refute several prior non-quantitative hypotheses that over-simplify or over-estimate the influence of underlying causes, thereby failing to capture the complex interplay between several factors. Our causal models also show a new connection between generalization and MI attacks via their shared causal factors. Our causal models have high predictive power ($0.90$), i.e., their analytical predictions match with observations in unseen experiments often, which makes analysis via them a pragmatic alternative.

arXiv comment26 pages, 15 figures; added CC-license block icons and links, typos corrected, added reference to Github

References in corpus (17)

• Comprehensive Privacy Analysis of Deep Learning: Passive and Active White-box Inference Attacks against Centralized and Federated Learning
• The Secret Sharer: Evaluating and Testing Unintended Memorization in Neural Networks
• Privacy Risks of Securing Machine Learning Models against Adversarial Examples
• Understanding Membership Inferences on Well-Generalized Learning Models
• Predicting the Generalization Gap in Deep Networks with Margin Distributions
• White-box vs Black-box: Bayes Optimal Strategies for Membership Inference
• Pearl's Calculus of Intervention Is Complete
• Analyzing Information Leakage of Updates to Natural Language Models
• Practical Blind Membership Inference Attack via Differential Comparisons
• Systematic Evaluation of Privacy Risks of Machine Learning Models
• Rethinking Bias-Variance Trade-off for Generalization of Neural Networks
• Node-Level Membership Inference Attacks Against Graph Neural Networks
• ML-Doctor: Holistic Risk Assessment of Inference Attacks Against Machine Learning Models
• ML Privacy Meter: Aiding Regulatory Compliance by Quantifying the Privacy Risks of Machine Learning
• On a Class of Bias-Amplifying Variables that Endanger Effect Estimates
• MemGuard: Defending against Black-Box Membership Inference Attacks via Adversarial Examples
• Learning Causal Graphs with Small Interventions

Cited by in corpus (1)

• Survey on AI Ethics: A Socio-technical Perspective