Fundamental Limits of Membership Inference Attacks on Machine Learning Models

Abstract

Membership inference attacks (MIA) can reveal whether a particular data point was part of the training dataset, potentially exposing sensitive information about individuals. This article provides theoretical guarantees by exploring the fundamental statistical limitations associated with MIAs on machine learning models at large. More precisely, we first derive the statistical quantity that governs the effectiveness and success of such attacks. We then theoretically prove that in a non-linear regression setting with overfitting learning procedures, attacks may have a high probability of success. Finally, we investigate several situations for which we provide bounds on this quantity of interest. Interestingly, our findings indicate that discretizing the data might enhance the learning procedure's security. Specifically, it is demonstrated to be limited by a constant, which quantifies the diversity of the underlying data distribution. We illustrate those results through simple simulations.

Accepted for publication in JMLR

References in corpus (11)

• Quantifying Memorization Across Neural Language Models
• Extracting Training Data from Diffusion Models
• Memorization Without Overfitting: Analyzing the Training Dynamics of Large Language Models
• ML Privacy Meter: Aiding Regulatory Compliance by Quantifying the Privacy Risks of Machine Learning
• The Privacy Onion Effect: Memorization is Relative
• Membership-Doctor: Comprehensive Assessment of Membership Inference Against Machine Learning Models
• Gradient Methods Provably Converge to Non-Robust Networks
• Bounding Membership Inference
• Measuring Unintended Memorisation of Unique Private Features in Neural Networks
• Analyzing Privacy Leakage in Machine Learning via Multiple Hypothesis Testing: A Lesson From Fano
• Provable Membership Inference Privacy