Backdoor Attacks on Federated Meta-Learning

Abstract

Federated learning allows multiple users to collaboratively train a shared classification model while preserving data privacy. This approach, where model updates are aggregated by a central server, was shown to be vulnerable to poisoning backdoor attacks: a malicious user can alter the shared model to arbitrarily classify specific inputs from a given class. In this paper, we analyze the effects of backdoor attacks on federated meta-learning, where users train a model that can be adapted to different sets of output classes using only a few examples. While the ability to adapt could, in principle, make federated learning frameworks more robust to backdoor attacks (when new training examples are benign), we find that even 1-shot~attacks can be very successful and persist after additional training. To address these vulnerabilities, we propose a defense mechanism inspired by matching networks, where the class of an input is predicted from the similarity of its features with a support set of labeled examples. By removing the decision logic from the model shared with the federation, success and persistence of backdoor attacks are greatly reduced.

arXiv comment13 pages, 19 figures, NeurIPS Workshop on Scalability, Privacy, and Security in Federated Learning (NeurIPS-SpicyFL), 2020

References in corpus (14)

• Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networks
• Prototypical Networks for Few-shot Learning
• Communication-Efficient Learning of Deep Networks from Decentralized Data
• Targeted Backdoor Attacks on Deep Learning Systems Using Data Poisoning
• BadNets: Identifying Vulnerabilities in the Machine Learning Model Supply Chain
• Matching Networks for One Shot Learning
• How To Backdoor Federated Learning
• Byzantine-Robust Distributed Learning: Towards Optimal Statistical Rates
• The Hidden Vulnerability of Distributed Learning in Byzantium
• On First-Order Meta-Learning Algorithms
• Spectral Signatures in Backdoor Attacks
• Mitigating Sybils in Federated Learning Poisoning
• Generalized Byzantine-tolerant SGD
• A Little Is Enough: Circumventing Defenses For Distributed Learning

Cited by in corpus (7)

• From Distributed Machine Learning to Federated Learning: A Survey
• FedML: A Research Library and Benchmark for Federated Machine Learning
• Survey on Federated Learning Threats: concepts, taxonomy on attacks and defences, experimental study and challenges
• Backdoor Attacks and Countermeasures on Deep Learning: A Comprehensive Review
• FedCV: A Federated Learning Framework for Diverse Computer Vision Tasks
• An Exploratory Analysis on Users' Contributions in Federated Learning
• Towards Bidirectional Protection in Federated Learning