Can You Really Backdoor Federated Learning?

Abstract

The decentralized nature of federated learning makes detecting and defending against adversarial attacks a challenging task. This paper focuses on backdoor attacks in the federated learning setting, where the goal of the adversary is to reduce the performance of the model on targeted tasks while maintaining good performance on the main task. Unlike existing works, we allow non-malicious clients to have correctly labeled samples from the targeted tasks. We conduct a comprehensive study of backdoor attacks and defenses for the EMNIST dataset, a real-life, user-partitioned, and non-iid dataset. We observe that in the absence of defenses, the performance of the attack largely depends on the fraction of adversaries present and the "complexity'' of the targeted task. Moreover, we show that norm clipping and "weak'' differential privacy mitigate the attacks without hurting the overall performance. We have implemented the attacks and defenses in TensorFlow Federated (TFF), a TensorFlow framework for federated learning. In open-sourcing our code, our goal is to encourage researchers to contribute new attacks and defenses and evaluate them on standard federated datasets.

To appear at the 2nd International Workshop on Federated Learning for Data Privacy and Confidentiality at NeurIPS 2019

References in corpus (11)

• Deep Learning with Differential Privacy
• Communication-Efficient Learning of Deep Networks from Decentralized Data
• Targeted Backdoor Attacks on Deep Learning Systems Using Data Poisoning
• Learning Differentially Private Recurrent Language Models
• Poisoning Attacks against Support Vector Machines
• LEAF: A Benchmark for Federated Settings
• Support Vector Machines under Adversarial Label Contamination
• Sever: A Robust Meta-Algorithm for Stochastic Optimization
• Learning with Bad Training Data via Iterative Trimmed Loss Minimization
• Backdoor Embedding in Convolutional Neural Network Models via Invisible Perturbation
• Data Poisoning against Differentially-Private Learners: Attacks and Defenses

Cited by in corpus (64)

• A Survey on Federated Learning Systems: Vision, Hype and Reality for Data Privacy and Protection
• From Distributed Machine Learning to Federated Learning: A Survey
• FedML: A Research Library and Benchmark for Federated Machine Learning
• Survey on Federated Learning Threats: concepts, taxonomy on attacks and defences, experimental study and challenges
• Ditto: Fair and Robust Federated Learning Through Personalization
• DeepSight: Mitigating Backdoor Attacks in Federated Learning Through Deep Model Inspection
• Learning to Detect Malicious Clients for Robust Federated Learning
• A Field Guide to Federated Optimization
• Wild Patterns Reloaded: A Survey of Machine Learning Security against Training Data Poisoning
• Federated Learning: Opportunities and Challenges
• Backdoor Attacks and Countermeasures on Deep Learning: A Comprehensive Review
• Attack of the Tails: Yes, You Really Can Backdoor Federated Learning
• Responsible and Regulatory Conform Machine Learning for Medicine: A Survey of Challenges and Solutions
• Mitigating Backdoor Attacks in Federated Learning
• Data Poisoning Attacks Against Federated Learning Systems
• A Survey on Vulnerability of Federated Learning: A Learning Algorithm Perspective
• FedCV: A Federated Learning Framework for Diverse Computer Vision Tasks
• PPFL: Privacy-preserving Federated Learning with Trusted Execution Environments
• FedScale: Benchmarking Model and System Performance of Federated Learning at Scale
• Reflection Backdoor: A Natural Backdoor Attack on Deep Neural Networks
• Backdoor attacks and defenses in feature-partitioned collaborative learning
• PASS: A Parameter Audit-based Secure and Fair Federated Learning Scheme against Free-Rider Attack
• Federated Learning Meets Multi-objective Optimization
• Turbo-Aggregate: Breaking the Quadratic Aggregation Barrier in Secure Federated Learning
• PEPPER: Empowering User-Centric Recommender Systems over Gossip Learning
• Learning from History for Byzantine Robust Optimization
• On the Outsized Importance of Learning Rates in Local Update Methods
• Resisting Backdoor Attacks in Federated Learning via Bidirectional Elections and Individual Perspective
• Robust Distributed Optimization With Randomly Corrupted Gradients
• Securing Federated Sensitive Topic Classification against Poisoning Attacks
• Distributed Momentum for Byzantine-resilient Learning
• You Can Backdoor Personalized Federated Learning
• Dynamic backdoor attacks against federated learning
• Byzantine-Robust Learning on Heterogeneous Datasets via Bucketing
• SPECTRE: Defending Against Backdoor Attacks Using Robust Statistics
• Towards Security Threats of Deep Learning Systems: A Survey
• ScionFL: Efficient and Robust Secure Quantized Aggregation
• Blockchain-Based Federated Learning in Mobile Edge Networks with Application in Internet of Vehicles
• Byzantine-Resilient Secure Federated Learning
• BlockFLA: Accountable Federated Learning via Hybrid Blockchain Architecture
• Hijack Vertical Federated Learning Models As One Party
• The More, the Better? A Study on Collaborative Machine Learning for DGA Detection
• PRECAD: Privacy-Preserving and Robust Federated Learning via Crypto-Aided Differential Privacy
• Preventing the Popular Item Embedding Based Attack in Federated Recommendations
• BACKDOORL: Backdoor Attack against Competitive Reinforcement Learning
• Mitigating Sybil Attacks on Differential Privacy based Federated Learning
• Enhancing Security and Privacy in Federated Learning using Low-Dimensional Update Representation and Proximity-Based Defense
• An Exploratory Analysis on Users' Contributions in Federated Learning
• Robust Federated Learning with Attack-Adaptive Aggregation
• One-Shot Federated Learning with Neuromorphic Processors
• Curse or Redemption? How Data Heterogeneity Affects the Robustness of Federated Learning
• ABC-FL: Anomalous and Benign client Classification in Federated Learning
• An Operator Splitting View of Federated Learning
• Towards Bidirectional Protection in Federated Learning
• Defending Against Diverse Attacks in Federated Learning Through Consensus-Based Bi-Level Optimization
• Robust Federated Learning by Mixture of Experts
• Security and Privacy Issues and Solutions in Federated Learning for Digital Healthcare
• Understanding the Interplay between Privacy and Robustness in Federated Learning
• Improving the Algorithm of Deep Learning with Differential Privacy
• Federated Learning from Small Datasets
• Combining Differential Privacy and Byzantine Resilience in Distributed SGD
• BadVFL: Backdoor Attacks in Vertical Federated Learning
• Toward Smart Security Enhancement of Federated Learning Networks
• Widen The Backdoor To Let More Attackers In