Addressing Class Imbalance in Federated Learning
arXiv:2008.06217
Abstract
Federated learning (FL) is a promising approach for training decentralized data located on local client devices while improving efficiency and privacy. However, the distribution and quantity of the training data on the clients' side may lead to significant challenges such as class imbalance and non-IID (non-independent and identically distributed) data, which could greatly impact the performance of the common model. While much effort has been devoted to helping FL models converge when encountering non-IID data, the imbalance issue has not been sufficiently addressed. In particular, as FL training is executed by exchanging gradients in an encrypted form, the training data is not completely observable to either clients or servers, and previous methods for class imbalance do not perform well for FL. Therefore, it is crucial to design new methods for detecting class imbalance in FL and mitigating its impact. In this work, we propose a monitoring scheme that can infer the composition of training data for each FL round, and design a new loss function -- \textbf{Ratio Loss} to mitigate the impact of the imbalance. Our experiments demonstrate the importance of acknowledging class imbalance and taking measures as early as possible in FL training, and the effectiveness of our method in mitigating the impact. Our method is shown to significantly outperform previous methods, while maintaining client privacy.
arXiv comment17 pages, Accept by AAAI 2021
References in corpus (19)
- Distributed Representations of Words and Phrases and their Compositionality
- Communication-Efficient Learning of Deep Networks from Decentralized Data
- A systematic study of the class imbalance problem in convolutional neural networks
- Federated Learning: Strategies for Improving Communication Efficiency
- Federated Learning for Mobile Keyboard Prediction
- Secure Federated Transfer Learning
- Horovod: fast and easy distributed deep learning in TensorFlow
- Differentially Private Federated Learning: A Client Level Perspective
- Distributed Federated Learning for Ultra-Reliable Low-Latency Vehicular Communications
- LEAF: A Benchmark for Federated Settings
- Federated Adversarial Domain Adaptation
- Class-Balanced Loss Based on Effective Number of Samples
- Updates-Leak: Data Set Inference and Reconstruction Attacks in Online Learning
- Private Federated Learning with Domain Adaptation
- Eavesdrop the Composition Proportion of Training Labels in Federated Learning
- Input Similarity from the Neural Network Perspective
- Data Poisoning Attacks on Federated Machine Learning
- Meta Feature Modulator for Long-tailed Recognition
- GRACE: Gradient Harmonized and Cascaded Labeling for Aspect-based Sentiment Analysis