Federated Learning with Fair Averaging
arXiv:2104.14937
Abstract
Fairness has emerged as a critical problem in federated learning (FL). In this work, we identify a cause of unfairness in FL -- conflicting gradients with large differences in the magnitudes. To address this issue, we propose the federated fair averaging (FedFV) algorithm to mitigate potential conflicts among clients before averaging their gradients. We first use the cosine similarity to detect gradient conflicts, and then iteratively eliminate such conflicts by modifying both the direction and the magnitude of the gradients. We further show the theoretical foundation of FedFV to mitigate the issue conflicting gradients and converge to Pareto stationary solutions. Extensive experiments on a suite of federated datasets confirm that FedFV compares favorably against state-of-the-art methods in terms of fairness, accuracy and efficiency. The source code is available at https://github.com/WwZzz/easyFL.
arXiv commentAccepted by IJCAI2021
References in corpus (15)
- Fashion-MNIST: a Novel Image Dataset for Benchmarking Machine Learning Algorithms
- Communication-Efficient Learning of Deep Networks from Decentralized Data
- Federated Learning with Non-IID Data
- On the Convergence of FedAvg on Non-IID Data
- Gradient Episodic Memory for Continual Learning
- Federated Optimization in Heterogeneous Networks
- Fair Resource Allocation in Federated Learning
- Ditto: Fair and Robust Federated Learning Through Personalization
- Local SGD Converges Fast and Communicates Little
- Gradient Surgery for Multi-Task Learning
- Mitigating Bias in Federated Learning
- A Reputation Mechanism Is All You Need: Collaborative Fairness and Adversarial Robustness in Federated Learning
- Orthogonal Gradient Descent for Continual Learning
- Fairness and Accuracy in Federated Learning
- Federated Learning Meets Multi-objective Optimization