Byzantines can also Learn from History: Fall of Centered Clipping in Federated Learning

Abstract

The increasing popularity of the federated learning (FL) framework due to its success in a wide range of collaborative learning tasks also induces certain security concerns. Among many vulnerabilities, the risk of Byzantine attacks is of particular concern, which refers to the possibility of malicious clients participating in the learning process. Hence, a crucial objective in FL is to neutralize the potential impact of Byzantine attacks and to ensure that the final model is trustable. It has been observed that the higher the variance among the clients' models/updates, the more space there is for Byzantine attacks to be hidden. As a consequence, by utilizing momentum, and thus, reducing the variance, it is possible to weaken the strength of known Byzantine attacks. The centered clipping (CC) framework has further shown that the momentum term from the previous iteration, besides reducing the variance, can be used as a reference point to neutralize Byzantine attacks better. In this work, we first expose vulnerabilities of the CC framework, and introduce a novel attack strategy that can circumvent the defences of CC and other robust aggregators and reduce their test accuracy up to %33 on best-case scenarios in image classification tasks. Then, we propose a new robust and fast defence mechanism that is effective against the proposed and other existing Byzantine attacks.

IEEE Transactions on Information Forensics and Security 2023

References in corpus (14)

• Fashion-MNIST: a Novel Image Dataset for Benchmarking Machine Learning Algorithms
• Targeted Backdoor Attacks on Deep Learning Systems Using Data Poisoning
• Poisoning Attacks against Support Vector Machines
• Federated Learning for Emoji Prediction in a Mobile Keyboard
• Abnormal Client Behavior Detection in Federated Learning
• SlowMo: Improving Communication-Efficient Distributed SGD with Slow Momentum
• Semi-Federated Learning for Collaborative Intelligence in Massive IoT Networks
• Dopamine: Differentially Private Federated Learning on Medical Data
• Robust Aggregation for Adaptive Privacy Preserving Federated Learning in Healthcare
• Semi-Federated Learning: Convergence Analysis and Optimization of A Hybrid Learning Framework
• Towards Communication-Learning Trade-off for Federated Learning at the Network Edge
• Byzantine Machine Learning Made Easy by Resilient Averaging of Momentums
• Fixing by Mixing: A Recipe for Optimal Byzantine ML under Heterogeneity
• Can Decentralized Learning be more robust than Federated Learning?