Eavesdrop the Composition Proportion of Training Labels in Federated Learning

Abstract

Federated learning (FL) has recently emerged as a new form of collaborative machine learning, where a common model can be learned while keeping all the training data on local devices. Although it is designed for enhancing the data privacy, we demonstrated in this paper a new direction in inference attacks in the context of FL, where valuable information about training data can be obtained by adversaries with very limited power. In particular, we proposed three new types of attacks to exploit this vulnerability. The first type of attack, Class Sniffing, can detect whether a certain label appears in training. The other two types of attacks can determine the quantity of each label, i.e., Quantity Inference attack determines the composition proportion of the training label owned by the selected clients in a single round, while Whole Determination attack determines that of the whole training process. We evaluated our attacks on a variety of tasks and datasets with different settings, and the corresponding results showed that our attacks work well generally. Finally, we analyzed the impact of major hyper-parameters to our attacks and discussed possible defenses.

arXiv comment15 pages, 10 figures, security conference

References in corpus (11)

• Scikit-learn: Machine Learning in Python
• Communication-Efficient Learning of Deep Networks from Decentralized Data
• Federated Learning: Strategies for Improving Communication Efficiency
• Comprehensive Privacy Analysis of Deep Learning: Passive and Active White-box Inference Attacks against Centralized and Federated Learning
• Federated Learning for Mobile Keyboard Prediction
• How To Backdoor Federated Learning
• Differentially Private Federated Learning: A Client Level Perspective
• Distributed Federated Learning for Ultra-Reliable Low-Latency Vehicular Communications
• Mitigating Sybils in Federated Learning Poisoning
• Updates-Leak: Data Set Inference and Reconstruction Attacks in Online Learning
• SparkNet: Training Deep Networks in Spark

Cited by in corpus (8)

• Survey on Federated Learning Threats: concepts, taxonomy on attacks and defences, experimental study and challenges
• DeepSight: Mitigating Backdoor Attacks in Federated Learning Through Deep Model Inspection
• A Field Guide to Federated Optimization
• Flamingo: Multi-Round Single-Server Secure Aggregation with Applications to Private Federated Learning
• ScionFL: Efficient and Robust Secure Quantized Aggregation
• Addressing Class Imbalance in Federated Learning
• An Exploratory Analysis on Users' Contributions in Federated Learning
• I3DOL: Incremental 3D Object Learning without Catastrophic Forgetting