Accurate and Fast Federated Learning via IID and Communication-Aware Grouping
arXiv:2012.04857
Abstract
Federated learning has emerged as a new paradigm of collaborative machine learning; however, it has also faced several challenges such as non-independent and identically distributed(IID) data and high communication cost. To this end, we propose a novel framework of IID and communication-aware group federated learning that simultaneously maximizes both accuracy and communication speed by grouping nodes based on data distributions and physical locations of the nodes. Furthermore, we provide a formal convergence analysis and an efficient optimization algorithm called FedAvg-IC. Experimental results show that, compared with the state-of-the-art algorithms, FedAvg-IC improved the test accuracy by up to 22.2% and simultaneously reduced the communication time to as small as 12%.
References in corpus (12)
- Fashion-MNIST: a Novel Image Dataset for Benchmarking Machine Learning Algorithms
- Communication-Efficient Learning of Deep Networks from Decentralized Data
- Federated Learning with Non-IID Data
- Federated Optimization: Distributed Machine Learning for On-Device Intelligence
- On the Convergence of FedAvg on Non-IID Data
- Federated Optimization in Heterogeneous Networks
- Communication-Efficient On-Device Machine Learning: Federated Distillation and Augmentation under Non-IID Private Data
- Jellyfish: Networking Data Centers Randomly
- LEAF: A Benchmark for Federated Settings
- Don't Use Large Mini-Batches, Use Local SGD
- Client-Edge-Cloud Hierarchical Federated Learning
- Hybrid-FL for Wireless Networks: Cooperative Learning Mechanism Using Non-IID Data