Source Inference Attacks in Federated Learning

Abstract

Federated learning (FL) has emerged as a promising privacy-aware paradigm that allows multiple clients to jointly train a model without sharing their private data. Recently, many studies have shown that FL is vulnerable to membership inference attacks (MIAs) that can distinguish the training members of the given model from the non-members. However, existing MIAs ignore the source of a training member, i.e., the information of which client owns the training member, while it is essential to explore source privacy in FL beyond membership privacy of examples from all clients. The leakage of source information can lead to severe privacy issues. For example, identification of the hospital contributing to the training of an FL model for COVID-19 pandemic can render the owner of a data record from this hospital more prone to discrimination if the hospital is in a high risk region. In this paper, we propose a new inference attack called source inference attack (SIA), which can derive an optimal estimation of the source of a training member. Specifically, we innovatively adopt the Bayesian perspective to demonstrate that an honest-but-curious server can launch an SIA to steal non-trivial source information of the training members without violating the FL protocol. The server leverages the prediction loss of local models on the training members to achieve the attack effectively and non-intrusively. We conduct extensive experiments on one synthetic and five real datasets to evaluate the key factors in an SIA, and the results show the efficacy of the proposed source inference attack.

arXiv commentThis paper has been accepted by ICDM 2021

References in corpus (19)

• Communication-Efficient Learning of Deep Networks from Decentralized Data
• Federated Learning with Non-IID Data
• Comprehensive Privacy Analysis of Deep Learning: Passive and Active White-box Inference Attacks against Centralized and Federated Learning
• On the Convergence of FedAvg on Non-IID Data
• How To Backdoor Federated Learning
• Learning Differentially Private Recurrent Language Models
• The Secret Sharer: Evaluating and Testing Unintended Memorization in Neural Networks
• Ensemble Distillation for Robust Model Fusion in Federated Learning
• Differentially Private Federated Learning: A Client Level Perspective
• FedMD: Heterogenous Federated Learning via Model Distillation
• FedML: A Research Library and Benchmark for Federated Machine Learning
• Protection Against Reconstruction and Its Applications in Private Federated Learning
• Deep Models Under the GAN: Information Leakage from Collaborative Deep Learning
• Evaluating Differentially Private Machine Learning in Practice
• Towards Demystifying Membership Inference Attacks
• Machine Learning Models that Remember Too Much
• Federated Model Distillation with Noise-Free Differential Privacy
• Membership Inference Attacks on Knowledge Graphs
• Secure Deep Graph Generation with Link Differential Privacy