How To Backdoor Federated Learning

Abstract

Federated learning enables thousands of participants to construct a deep learning model without sharing their private training data with each other. For example, multiple smartphones can jointly train a next-word predictor for keyboards without revealing what individual users type. We demonstrate that any participant in federated learning can introduce hidden backdoor functionality into the joint global model, e.g., to ensure that an image classifier assigns an attacker-chosen label to images with certain features, or that a word predictor completes certain sentences with an attacker-chosen word. We design and evaluate a new model-poisoning methodology based on model replacement. An attacker selected in a single round of federated learning can cause the global model to immediately reach 100% accuracy on the backdoor task. We evaluate the attack under different assumptions for the standard federated-learning tasks and show that it greatly outperforms data poisoning. Our generic constrain-and-scale technique also evades anomaly detection-based defenses by incorporating the evasion into the attacker's loss function during training.

References in corpus (32)

• Distilling the Knowledge in a Neural Network
• Overcoming catastrophic forgetting in neural networks
• Deep Learning with Differential Privacy
• Federated Learning: Strategies for Improving Communication Efficiency
• Federated Learning for Mobile Keyboard Prediction
• Targeted Backdoor Attacks on Deep Learning Systems Using Data Poisoning
• BadNets: Identifying Vulnerabilities in the Machine Learning Model Supply Chain
• Towards Federated Learning at Scale: System Design
• An Empirical Investigation of Catastrophic Forgetting in Gradient-Based Neural Networks
• Learning Differentially Private Recurrent Language Models
• Poisoning Attacks against Support Vector Machines
• Byzantine-Robust Distributed Learning: Towards Optimal Statistical Rates
• Revisiting Distributed Synchronous SGD
• The Hidden Vulnerability of Distributed Learning in Byzantium
• The Secret Sharer: Evaluating and Testing Unintended Memorization in Neural Networks
• Differentially Private Federated Learning: A Client Level Perspective
• Private federated learning on vertically partitioned data via entity resolution and additively homomorphic encryption
• Analyzing Federated Learning through an Adversarial Lens
• Spectral Signatures in Backdoor Attacks
• Mitigating Sybils in Federated Learning Poisoning
• Scalable Private Learning with PATE
• Detecting Backdoor Attacks on Deep Neural Networks by Activation Clustering
• Tying Word Vectors and Word Classifiers: A Loss Framework for Language Modeling
• Generalized Byzantine-tolerant SGD
• Distributed Statistical Machine Learning in Adversarial Settings: Byzantine Gradient Descent
• PoTrojan: powerful neural-level trojan designs in deep learning models
• Contamination Attacks and Mitigation in Multi-Party Machine Learning
• Fine-Pruning: Defending Against Backdooring Attacks on Deep Neural Networks
• Distributed Training with Heterogeneous Data: Bridging Median- and Mean-Based Algorithms
• Machine Learning Models that Remember Too Much
• Asynchronous Byzantine Machine Learning (the case of SGD)
• Learning Discrete Distributions from Untrusted Batches

Cited by in corpus (183)

• A Survey on Federated Learning Systems: Vision, Hype and Reality for Data Privacy and Protection
• On the Convergence of FedAvg on Non-IID Data
• Towards Federated Learning at Scale: System Design
• A Survey on Distributed Machine Learning
• Federated Machine Learning: Concept and Applications
• Precision Health Data: Requirements, Challenges and Existing Techniques for Data Security and Privacy
• Analyzing Federated Learning through an Adversarial Lens
• From Distributed Machine Learning to Federated Learning: A Survey
• Mitigating Sybils in Federated Learning Poisoning
• FedML: A Research Library and Benchmark for Federated Machine Learning
• Distillation-Based Semi-Supervised Federated Learning for Communication-Efficient Collaborative Training with Non-IID Private Data
• LEAF: A Benchmark for Federated Settings
• Februus: Input Purification Defense Against Trojan Attacks on Deep Neural Network Systems
• Threats to Federated Learning: A Survey
• Ditto: Fair and Robust Federated Learning Through Personalization
• Federated Collaborative Filtering for Privacy-Preserving Personalized Recommendation System
• FMore: An Incentive Scheme of Multi-dimensional Auction for Federated Learning in MEC
• Federated Neural Collaborative Filtering
• FedPD: A Federated Learning Framework with Optimal Rates and Adaptivity to Non-IID Data
• DeepSight: Mitigating Backdoor Attacks in Federated Learning Through Deep Model Inspection
• Learning to Detect Malicious Clients for Robust Federated Learning
• Local Model Poisoning Attacks to Byzantine-Robust Federated Learning
• Differential Privacy-enabled Federated Learning for Sensitive Health Data
• Byzantine-Robust Federated Machine Learning through Adaptive Model Averaging
• Blockchain-based Federated Learning with Secure Aggregation in Trusted Execution Environment for Internet-of-Things
• Federated Learning in Mobile Edge Networks: A Comprehensive Survey
• Efficient Dropout-resilient Aggregation for Privacy-preserving Machine Learning
• Edge Intelligence: Architectures, Challenges, and Applications
• Federated Continual Learning via Knowledge Fusion: A Survey
• Blockchain-based Federated Learning with SMPC Model Verification Against Poisoning Attack for Healthcare Systems
• Abnormal Client Behavior Detection in Federated Learning
• Attack of the Tails: Yes, You Really Can Backdoor Federated Learning
• Blind Backdoors in Deep Learning Models
• Machine Learning for Security in Vehicular Networks: A Comprehensive Survey
• A Framework for Evaluating Gradient Leakage Attacks in Federated Learning
• Rethinking the Trigger of Backdoor Attack
• Cronus: Robust and Heterogeneous Collaborative Learning with Black-Box Knowledge Transfer
• An Experimental Study of Byzantine-Robust Aggregation Schemes in Federated Learning
• STRIP: A Defence Against Trojan Attacks on Deep Neural Networks
• Evaluating Differentially Private Machine Learning in Practice
• Active Federated Learning
• Flamingo: Multi-Round Single-Server Secure Aggregation with Applications to Private Federated Learning
• FedRecAttack: Model Poisoning Attack to Federated Recommendation
• Privacy-Preserving Machine Learning: Methods, Challenges and Directions
• Distilled One-Shot Federated Learning
• Demon in the Variant: Statistical Analysis of DNNs for Robust Backdoor Contamination Detection
• On the Effectiveness of Mitigating Data Poisoning Attacks with Gradient Shaping
• Free-riders in Federated Learning: Attacks and Defenses
• Salvaging Federated Learning by Local Adaptation
• Federated Learning for Healthcare Informatics
• A Systematic Survey of Blockchained Federated Learning
• Federated Unlearning: A Survey on Methods, Design Guidelines, and Evaluation Metrics
• FedLab: A Flexible Federated Learning Framework
• FjORD: Fair and Accurate Federated Learning under heterogeneous targets with Ordered Dropout
• You Autocomplete Me: Poisoning Vulnerabilities in Neural Code Completion
• Decentralized Deep Learning for Multi-Access Edge Computing: A Survey on Communication Efficiency and Trustworthiness
• Can Adversarial Weight Perturbations Inject Neural Backdoors?
• Mitigating Backdoor Attacks in Federated Learning
• The Internet of Federated Things (IoFT): A Vision for the Future and In-depth Survey of Data-driven Approaches for Federated Learning
• Fall of Empires: Breaking Byzantine-tolerant SGD by Inner Product Manipulation
• Attack-Resistant Federated Learning with Residual-based Reweighting
• Dynamic Defense Against Byzantine Poisoning Attacks in Federated Learning
• Data Poisoning Attacks Against Federated Learning Systems
• Shielding Federated Learning: Robust Aggregation with Adaptive Client Selection
• Backdoor Scanning for Deep Neural Networks through K-Arm Optimization
• No Peek: A Survey of private distributed deep learning
• Towards Communication-efficient and Attack-Resistant Federated Edge Learning for Industrial Internet of Things
• Poisoning Deep Learning Based Recommender Model in Federated Learning Scenarios
• A Little Is Enough: Circumventing Defenses For Distributed Learning
• KD3A: Unsupervised Multi-Source Decentralized Domain Adaptation via Knowledge Distillation
• FedCV: A Federated Learning Framework for Diverse Computer Vision Tasks
• SAFELearning: Enable Backdoor Detectability In Federated Learning With Secure Aggregation
• PPFL: Privacy-preserving Federated Learning with Trusted Execution Environments
• Challenges of Privacy-Preserving Machine Learning in IoT
• Federated User Representation Learning
• Backdoor attacks and defenses in feature-partitioned collaborative learning
• FLTrust: Byzantine-robust Federated Learning via Trust Bootstrapping
• Eavesdrop the Composition Proportion of Training Labels in Federated Learning
• Spinning Language Models: Risks of Propaganda-As-A-Service and Countermeasures
• Data-Quality Based Scheduling for Federated Edge Learning
• Distributed Training with Heterogeneous Data: Bridging Median- and Mean-Based Algorithms
• Federated Learning Meets Multi-objective Optimization
• Towards a Robust and Trustworthy Machine Learning System Development: An Engineering Perspective
• Secure Federated Submodel Learning
• The Threat of Adversarial Attacks on Machine Learning in Network Security -- A Survey
• Decentralized Learning of Generative Adversarial Networks from Non-iid Data
• A Federated Learning Approach for Mobile Packet Classification
• BAFFLE : Blockchain Based Aggregator Free Federated Learning
• Natural Backdoor Attack on Text Data
• Towards Reliable Participation in UAV-Enabled Federated Edge Learning on Non-IID Data
• FedEval: A Holistic Evaluation Framework for Federated Learning
• TrojanZoo: Towards Unified, Holistic, and Practical Evaluation of Neural Backdoors
• On the Outsized Importance of Learning Rates in Local Update Methods
• Backdoor Attacks on Federated Meta-Learning
• A backdoor attack against LSTM-based text classification systems
• Resisting Backdoor Attacks in Federated Learning via Bidirectional Elections and Individual Perspective
• Robust Federated Recommendation System
• Towards Crowdsourced Training of Large Neural Networks using Decentralized Mixture-of-Experts
• On Safeguarding Privacy and Security in the Framework of Federated Learning
• Robust Distributed Optimization With Randomly Corrupted Gradients
• Byzantines can also Learn from History: Fall of Centered Clipping in Federated Learning
• Shielding Collaborative Learning: Mitigating Poisoning Attacks through Client-Side Detection
• FedV: Privacy-Preserving Federated Learning over Vertically Partitioned Data
• Securing Federated Sensitive Topic Classification against Poisoning Attacks
• Design and Evaluation of a Multi-Domain Trojan Detection Method on Deep Neural Networks
• CAFE: Catastrophic Data Leakage in Vertical Federated Learning
• Data Poisoning Attacks on Federated Machine Learning
• LoAdaBoost: loss-based AdaBoost federated machine learning with reduced computational complexity on IID and non-IID intensive care data
• FedDebug: Systematic Debugging for Federated Learning Applications
• FedLoc: Federated Learning Framework for Data-Driven Cooperative Localization and Location Data Processing
• T-Miner: A Generative Approach to Defend Against Trojan Attacks on DNN-based Text Classification
• Byzantine-Robust and Privacy-Preserving Framework for FedML
• Dynamic backdoor attacks against federated learning
• Light Can Hack Your Face! Black-box Backdoor Attack on Face Recognition Systems
• Detecting Backdoors in Neural Networks Using Novel Feature-Based Anomaly Detection
• Byzantine-Robust Learning on Heterogeneous Datasets via Bucketing
• Optimal query complexity for private sequential learning against eavesdropping
• FedGL: Federated Graph Learning Framework with Global Self-Supervision
• Towards Security Threats of Deep Learning Systems: A Survey
• Blockchain-Based Federated Learning in Mobile Edge Networks with Application in Internet of Vehicles
• Poisoning Attacks to Local Differential Privacy Protocols for Key-Value Data
• Federated Learning System without Model Sharing through Integration of Dimensional Reduced Data Representations
• BlockFLA: Accountable Federated Learning via Hybrid Blockchain Architecture
• Characterizing Impacts of Heterogeneity in Federated Learning upon Large-Scale Smartphone Data
• Sleeper Agent: Scalable Hidden Trigger Backdoors for Neural Networks Trained from Scratch
• Hijack Vertical Federated Learning Models As One Party
• The More, the Better? A Study on Collaborative Machine Learning for DGA Detection
• Differentially Private Federated Learning with Laplacian Smoothing
• Stochastic Channel-Based Federated Learning for Medical Data Privacy Preserving
• Anonymizing Data for Privacy-Preserving Federated Learning
• Auxo: Efficient Federated Learning via Scalable Client Clustering
• Towards On-Device Federated Learning: A Direct Acyclic Graph-based Blockchain Approach
• Wireless for Machine Learning
• Revealing Perceptible Backdoors, without the Training Set, via the Maximum Achievable Misclassification Fraction Statistic
• Security of Deep Learning Methodologies: Challenges and Opportunities
• Mitigating backdoor attacks in LSTM-based Text Classification Systems by Backdoor Keyword Identification
• Mitigating Sybil Attacks on Differential Privacy based Federated Learning
• OCTOPUS: Overcoming Performance andPrivatization Bottlenecks in Distributed Learning
• Privacy Assessment of Federated Learning using Private Personalized Layers
• Towards Fast and Stable Federated Learning: Confronting Heterogeneity via Knowledge Anchor
• Federated Learning in Adversarial Settings
• Privacy Preserving Stochastic Channel-Based Federated Learning with Neural Network Pruning
• WAFFLE: Watermarking in Federated Learning
• Source Inference Attacks in Federated Learning
• Trends and Advancements in Deep Neural Network Communication
• Policy-Based Federated Learning
• Robust Federated Learning with Attack-Adaptive Aggregation
• SLSGD: Secure and Efficient Distributed On-device Machine Learning
• NTD: Non-Transferability Enabled Backdoor Detection
• Resource-constrained Federated Edge Learning with Heterogeneous Data: Formulation and Analysis
• Federated Unlearning
• Citadel: Protecting Data Privacy and Model Confidentiality for Collaborative Learning with SGX
• Curse or Redemption? How Data Heterogeneity Affects the Robustness of Federated Learning
• One-Shot Federated Learning with Neuromorphic Processors
• A General Framework for Defending Against Backdoor Attacks via Influence Graph
• Qu-ANTI-zation: Exploiting Quantization Artifacts for Achieving Adversarial Outcomes
• State-of-the-art Techniques in Deep Edge Intelligence
• Privacy-Preserving Federated Learning on Partitioned Attributes
• MANDERA: Malicious Node Detection in Federated Learning via Ranking
• Privacy Threats Analysis to Secure Federated Learning
• Client Adaptation improves Federated Learning with Simulated Non-IID Clients
• A Novel Privacy-Preserved Recommender System Framework based on Federated Learning
• Towards Bidirectional Protection in Federated Learning
• Adaptive Distillation for Decentralized Learning from Heterogeneous Clients
• Pocket Diagnosis: Secure Federated Learning against Poisoning Attack in the Cloud
• ABC-FL: Anomalous and Benign client Classification in Federated Learning
• An Operator Splitting View of Federated Learning
• Reaching Data Confidentiality and Model Accountability on the CalTrain
• Spinning Sequence-to-Sequence Models with Meta-Backdoors
• SecEL: Privacy-Preserving, Verifiable and Fault-Tolerant Edge Learning for Autonomous Vehicles
• Security and Privacy Issues and Solutions in Federated Learning for Digital Healthcare
• Federated Learning with Unreliable Clients: Performance Analysis and Mechanism Design
• 10 Security and Privacy Problems in Large Foundation Models
• On Provable Backdoor Defense in Collaborative Learning
• FedProf: Selective Federated Learning with Representation Profiling
• Simeon -- Secure Federated Machine Learning Through Iterative Filtering
• Robust Federated Learning by Mixture of Experts
• BadVFL: Backdoor Attacks in Vertical Federated Learning
• LINDT: Tackling Negative Federated Learning with Local Adaptation
• FLBench: A Benchmark Suite for Federated Learning
• FedNNNN: Norm-Normalized Neural Network Aggregation for Fast and Accurate Federated Learning
• PipAttack: Poisoning Federated Recommender Systems forManipulating Item Promotion
• GraphFederator: Federated Visual Analysis for Multi-party Graphs