Byzantine-Robust Federated Machine Learning through Adaptive Model Averaging

Abstract

Federated learning enables training collaborative machine learning models at scale with many participants whilst preserving the privacy of their datasets. Standard federated learning techniques are vulnerable to Byzantine failures, biased local datasets, and poisoning attacks. In this paper we introduce Adaptive Federated Averaging, a novel algorithm for robust federated learning that is designed to detect failures, attacks, and bad updates provided by participants in a collaborative model. We propose a Hidden Markov Model to model and learn the quality of model updates provided by each participant during training. In contrast to existing robust federated learning schemes, we propose a robust aggregation rule that detects and discards bad or malicious local model updates at each training iteration. This includes a mechanism that blocks unwanted participants, which also increases the computational and communication efficiency. Our experimental evaluation on 4 real datasets show that our algorithm is significantly more robust to faulty, noisy and malicious participants, whilst being computationally more efficient than other state-of-the-art robust federated learning methods such as Multi-KRUM and coordinate-wise median.

References in corpus (11)

• Very Deep Convolutional Networks for Large-Scale Image Recognition
• Fashion-MNIST: a Novel Image Dataset for Benchmarking Machine Learning Algorithms
• Communication-Efficient Learning of Deep Networks from Decentralized Data
• Federated Learning: Strategies for Improving Communication Efficiency
• Federated Optimization: Distributed Machine Learning for On-Device Intelligence
• How To Backdoor Federated Learning
• Poisoning Attacks against Support Vector Machines
• Byzantine-Robust Distributed Learning: Towards Optimal Statistical Rates
• The Hidden Vulnerability of Distributed Learning in Byzantium
• Zeno: Distributed Stochastic Gradient Descent with Suspicion-based Fault-tolerance
• Fall of Empires: Breaking Byzantine-tolerant SGD by Inner Product Manipulation

Cited by in corpus (20)

• From Distributed Machine Learning to Federated Learning: A Survey
• Survey on Federated Learning Threats: concepts, taxonomy on attacks and defences, experimental study and challenges
• DeepSight: Mitigating Backdoor Attacks in Federated Learning Through Deep Model Inspection
• A Decentralized Federated Learning Framework via Committee Mechanism with Convergence Guarantee
• Secure Byzantine-Robust Machine Learning
• Backdoor attacks and defenses in feature-partitioned collaborative learning
• FLTrust: Byzantine-robust Federated Learning via Trust Bootstrapping
• Robust Aggregation for Adaptive Privacy Preserving Federated Learning in Healthcare
• AGRAMPLIFIER: Defending Federated Learning Against Poisoning Attacks Through Local Update Amplification
• A Four-Pronged Defense Against Byzantine Attacks in Federated Learning
• Distributed Momentum for Byzantine-resilient Learning
• 2CP: Decentralized Protocols to Transparently Evaluate Contributivity in Blockchain Federated Learning Environments
• Byzantine-resilient Decentralized Stochastic Gradient Descent
• Robust Federated Learning with Attack-Adaptive Aggregation
• Data-Free Evaluation of User Contributions in Federated Learning
• Federated Learning via Plurality Vote
• FedRAD: Federated Robust Adaptive Distillation
• Federated Learning with Unreliable Clients: Performance Analysis and Mechanism Design
• Bristle: Decentralized Federated Learning in Byzantine, Non-i.i.d. Environments
• Toward Smart Security Enhancement of Federated Learning Networks