Perfectly Accurate Membership Inference by a Dishonest Central Server in Federated Learning

Abstract

Federated Learning is expected to provide strong privacy guarantees, as only gradients or model parameters but no plain text training data is ever exchanged either between the clients or between the clients and the central server. In this paper, we challenge this claim by introducing a simple but still very effective membership inference attack algorithm, which relies only on a single training step. In contrast to the popular honest-but-curious model, we investigate a framework with a dishonest central server. Our strategy is applicable to models with ReLU activations and uses the properties of this activation function to achieve perfect accuracy. Empirical evaluation on visual classification tasks with MNIST, CIFAR10, CIFAR100 and CelebA datasets show that our method provides perfect accuracy in identifying one sample in a training set with thousands of samples. Occasional failures of our method lead us to discover duplicate images in the CIFAR100 and CelebA datasets.

accepted for publication in IEEE Transactions on Dependable and Secure Computing

References in corpus (7)

• Private federated learning on vertically partitioned data via entity resolution and additively homomorphic encryption
• Practical Secure Aggregation for Federated Learning on User-Held Data
• A Framework for Evaluating Gradient Leakage Attacks in Federated Learning
• When the Curious Abandon Honesty: Federated Learning Is Not Private
• Fishing for User Data in Large-Batch Federated Learning via Gradient Magnification
• Gradient Disaggregation: Breaking Privacy in Federated Learning by Reconstructing the User Participant Matrix
• Robbing the Fed: Directly Obtaining Private Data in Federated Learning with Modified Models