Defending Against Diverse Attacks in Federated Learning Through Consensus-Based Bi-Level Optimization

Abstract

Adversarial attacks pose significant challenges in many machine learning applications, particularly in the setting of distributed training and federated learning, where malicious agents seek to corrupt the training process with the goal of jeopardizing and compromising the performance and reliability of the final models. In this paper, we address the problem of robust federated learning in the presence of such attacks by formulating the training task as a bi-level optimization problem. We conduct a theoretical analysis of the resilience of consensus-based bi-level optimization (CB${}^2$O), an interacting multi-particle metaheuristic optimization method, in adversarial settings. Specifically, we provide a global convergence analysis of CB${}^2$O in mean-field law in the presence of malicious agents, demonstrating the robustness of CB${}^2$O against a diverse range of attacks. Thereby, we offer insights into how specific hyperparameter choices enable to mitigate adversarial effects. On the practical side, we extend CB${}^2$O to the clustered federated learning setting by proposing FedCB${}^2$O, a novel interacting multi-particle system, and design a practical algorithm that addresses the demands of real-world applications. Extensive experiments demonstrate the robustness of the FedCB${}^2$O algorithm against label-flipping attacks in decentralized clustered federated learning scenarios, showcasing its effectiveness in practical contexts.

References in corpus (33)

• Federated Optimization: Distributed Machine Learning for On-Device Intelligence
• Comprehensive Privacy Analysis of Deep Learning: Passive and Active White-box Inference Attacks against Centralized and Federated Learning
• Targeted Backdoor Attacks on Deep Learning Systems Using Data Poisoning
• Byzantine-Robust Distributed Learning: Towards Optimal Statistical Rates
• Decentralized Federated Learning: Fundamentals, State of the Art, Frameworks, Trends, and Challenges
• Analyzing Federated Learning through an Adversarial Lens
• Can You Really Backdoor Federated Learning?
• Multi-Center Federated Learning: Clients Clustering for Better Personalization
• Multi-Center Federated Learning: Clients Clustering for Better Personalization
• Survey on Federated Learning Threats: concepts, taxonomy on attacks and defences, experimental study and challenges
• Federated Variance-Reduced Stochastic Gradient Descent with Robustness to Byzantine Attacks
• Inverting Gradients -- How easy is it to break privacy in federated learning?
• Local Model Poisoning Attacks to Byzantine-Robust Federated Learning
• Decentralized Federated Learning: A Survey on Security and Privacy
• A consensus-based model for global optimization and its mean-field limit
• On the Random Batch Method for second order interacting particle systems
• Attack of the Tails: Yes, You Really Can Backdoor Federated Learning
• Optimal Client Sampling for Federated Learning
• Fedstellar: A Platform for Decentralized Federated Learning
• On the Global Convergence of Particle Swarm Optimization Methods
• Consensus-Based Optimization Methods Converge Globally
• Decentralized federated learning of deep neural networks on non-iid data
• On the Convergence of Clustered Federated Learning
• Convergence of Anisotropic Consensus-Based Optimization in Mean-Field Law
• Leveraging Memory Effects and Gradient Information in Consensus-Based Optimization: On Global Convergence in Mean-Field Law
• Free-rider Attacks on Model Aggregation in Federated Learning
• A consensus-based global optimization method for high dimensional machine learning problems
• FedCBO: Reaching Group Consensus in Clustered Federated Learning through Consensus-based Optimization
• CBX: Python and Julia packages for consensus-based interacting particle methods
• FedSoft: Soft Clustered Federated Learning with Proximal Local Updating
• Swarm Learning: A Survey of Concepts, Applications, and Trends
• SPAM: Stochastic Proximal Point Method with Momentum Variance Reduction for Non-convex Cross-Device Federated Learning
• CB${}^2$O: Consensus-Based Bi-Level Optimization