Asynchronous Federated Learning with Reduced Number of Rounds and with Differential Privacy from Less Aggregated Gaussian Noise

Abstract

The feasibility of federated learning is highly constrained by the server-clients infrastructure in terms of network communication. Most newly launched smartphones and IoT devices are equipped with GPUs or sufficient computing hardware to run powerful AI models. However, in case of the original synchronous federated learning, client devices suffer waiting times and regular communication between clients and server is required. This implies more sensitivity to local model training times and irregular or missed updates, hence, less or limited scalability to large numbers of clients and convergence rates measured in real time will suffer. We propose a new algorithm for asynchronous federated learning which eliminates waiting times and reduces overall network communication - we provide rigorous theoretical analysis for strongly convex objective functions and provide simulation results. By adding Gaussian noise we show how our algorithm can be made differentially private -- new theorems show how the aggregated added Gaussian noise is significantly reduced.

References in corpus (11)

• Deep Learning with Differential Privacy
• Federated Learning: Strategies for Improving Communication Efficiency
• Federated Optimization: Distributed Machine Learning for On-Device Intelligence
• Towards Federated Learning at Scale: System Design
• Revisiting Distributed Synchronous SGD
• Communication-Efficient Federated Deep Learning with Asynchronous Model Update and Temporally Weighted Aggregation
• Federated Optimization in Heterogeneous Networks
• Asynchronous Federated Optimization
• Local SGD Converges Fast and Communicates Little
• Asynchronous Stochastic Gradient Descent with Delay Compensation
• A Distributed Synchronous SGD Algorithm with Global Top-$k$ Sparsification for Low Bandwidth Networks

Cited by in corpus (5)

• Federated Learning: Opportunities and Challenges
• Asynchronous Federated Learning on Heterogeneous Devices: A Survey
• Federated Learning with Buffered Asynchronous Aggregation
• Secure Aggregation for Buffered Asynchronous Federated Learning
• AutoFL: Enabling Heterogeneity-Aware Energy Efficient Federated Learning