Quantized and Asynchronous Federated Learning

Abstract

Recent advances in federated learning have shown that asynchronous variants can be faster and more scalable than their synchronous counterparts. However, their design does not include quantization, which is necessary in practice to deal with the communication bottleneck. To bridge this gap, we develop a novel algorithm, Quantized Asynchronous Federated Learning (QAFeL), which introduces a hidden-state quantization scheme to avoid the error propagation caused by direct quantization. QAFeL also includes a buffer to aggregate client updates, ensuring scalability and compatibility with techniques such as secure aggregation. Furthermore, we prove that QAFeL achieves an $\mathcal{O}(1/\sqrt{T})$ ergodic convergence rate for stochastic gradient descent on non-convex objectives, which is the optimal order of complexity, without requiring bounded gradients or uniform client arrivals. We also prove that the cross-term error between staleness and quantization only affects the higher-order error terms. We validate our theoretical findings on standard benchmarks.

References in corpus (7)

• Measuring the Effects of Non-Identical Data Distribution for Federated Visual Classification
• D-ADMM: A Communication-Efficient Distributed Algorithm For Separable Optimization
• UVeQFed: Universal Vector Quantization for Federated Learning
• Fast Federated Learning by Balancing Communication Trade-Offs
• Joint Privacy Enhancement and Quantization in Federated Learning
• A Better Alternative to Error Feedback for Communication-Efficient Distributed Learning
• Gossiped and Quantized Online Multi-Kernel Learning