Federated Learning Meets Multi-objective Optimization

Abstract

Federated learning has emerged as a promising, massively distributed way to train a joint deep model over large amounts of edge devices while keeping private user data strictly on device. In this work, motivated from ensuring fairness among users and robustness against malicious adversaries, we formulate federated learning as multi-objective optimization and propose a new algorithm FedMGDA+ that is guaranteed to converge to Pareto stationary solutions. FedMGDA+ is simple to implement, has fewer hyperparameters to tune, and refrains from sacrificing the performance of any participating user. We establish the convergence properties of FedMGDA+ and point out its connections to existing approaches. Extensive experiments on a variety of datasets confirm that FedMGDA+ compares favorably against state-of-the-art.

Accepted at IEEE Transactions on Network Science and Engineering 2022

References in corpus (12)

• Fashion-MNIST: a Novel Image Dataset for Benchmarking Machine Learning Algorithms
• How To Backdoor Federated Learning
• Federated Optimization in Heterogeneous Networks
• Can You Really Backdoor Federated Learning?
• Three Approaches for Personalization with Applications to Federated Learning
• Agnostic Federated Learning
• Bayesian Nonparametric Federated Learning of Neural Networks
• Federated Learning with Matched Averaging
• FedSplit: An algorithmic framework for fast federated optimization
• Pareto Multi-Task Learning
• Faster On-Device Training Using New Federated Momentum Algorithm
• Multi-objective training of Generative Adversarial Networks with multiple discriminators

Cited by in corpus (10)

• From Distributed Machine Learning to Federated Learning: A Survey
• Ditto: Fair and Robust Federated Learning Through Personalization
• A Field Guide to Federated Optimization
• Joint Local Relational Augmentation and Global Nash Equilibrium for Federated Learning with Non-IID Data
• FedPrune: Towards Inclusive Federated Learning
• On Large-Cohort Training for Federated Learning
• Federated Learning with Fair Averaging
• An Operator Splitting View of Federated Learning
• FedRAD: Federated Robust Adaptive Distillation
• Federating for Learning Group Fair Models