Correlation inference attacks against machine learning models

Abstract

Despite machine learning models being widely used today, the relationship between a model and its training dataset is not well understood. We explore correlation inference attacks, whether and when a model leaks information about the correlations between the input variables of its training dataset. We first propose a model-less attack, where an adversary exploits the spherical parametrization of correlation matrices alone to make an informed guess. Second, we propose a model-based attack, where an adversary exploits black-box model access to infer the correlations using minimal and realistic assumptions. Third, we evaluate our attacks against logistic regression and multilayer perceptron models on three tabular datasets and show the models to leak correlations. We finally show how extracted correlations can be used as building blocks for attribute inference attacks and enable weaker adversaries. Our results raise fundamental questions on what a model does and should remember from its training set.

Published in Science Advances. This version contains both the main paper and supplementary material. There are minor editorial differences between this version and the published version. The first two authors contributed equally

References in corpus (10)

• Scikit-learn: Machine Learning in Python
• PyTorch: An Imperative Style, High-Performance Deep Learning Library
• Comprehensive Privacy Analysis of Deep Learning: Passive and Active White-box Inference Attacks against Centralized and Federated Learning
• Differentially Private Empirical Risk Minimization
• Extracting Training Data from Large Language Models
• Algorithms that Remember: Model Inversion Attacks and Data Protection Law
• Label-Only Membership Inference Attacks
• Evaluating Differentially Private Machine Learning in Practice
• Leakage of Dataset Properties in Multi-Party Machine Learning
• Are Your Sensitive Attributes Private? Novel Model Inversion Attribute Inference Attacks on Classification Models