Towards Flexible Device Participation in Federated Learning
arXiv:2006.06954
Abstract
Traditional federated learning algorithms impose strict requirements on the participation rates of devices, which limit the potential reach of federated learning. This paper extends the current learning paradigm to include devices that may become inactive, compute incomplete updates, and depart or arrive in the middle of training. We derive analytical results to illustrate how allowing more flexible device participation can affect the learning convergence when data is not independently and identically distributed (non-IID). We then propose a new federated aggregation scheme that converges even when devices may be inactive or return incomplete updates. We also study how the learning process can adapt to early departures or late arrivals, and analyze their impacts on the convergence.
References in corpus (11)
- Communication-Efficient Learning of Deep Networks from Decentralized Data
- Federated Learning: Strategies for Improving Communication Efficiency
- Towards Federated Learning at Scale: System Design
- Federated Optimization in Heterogeneous Networks
- Expanding the Reach of Federated Learning by Reducing Client Resource Requirements
- Variational Federated Multi-Task Learning
- The Non-IID Data Quagmire of Decentralized Machine Learning
- Client-Edge-Cloud Hierarchical Federated Learning
- Characterizing Impacts of Heterogeneity in Federated Learning upon Large-Scale Smartphone Data
- Adaptive Gradient Sparsification for Efficient Federated Learning: An Online Learning Approach
- Machine Learning on Volatile Instances