Secure Aggregation with Heterogeneous Quantization in Federated Learning

Abstract

Secure model aggregation across many users is a key component of federated learning systems. The state-of-the-art protocols for secure model aggregation, which are based on additive masking, require all users to quantize their model updates to the same level of quantization. This severely degrades their performance due to lack of adaptation to available bandwidth at different users. We propose three schemes that allow secure model aggregation while using heterogeneous quantization. This enables the users to adjust their quantization proportional to their available bandwidth, which can provide a substantially better trade-off between the accuracy of training and the communication time. The proposed schemes are based on a grouping strategy by partitioning the network into groups, and partitioning the local model updates of users into segments. Instead of applying aggregation protocol to the entire local model update vector, it is applied on segments with specific coordination between users. We theoretically evaluate the quantization error for our schemes, and also demonstrate how our schemes can be utilized to overcome Byzantine users.

References in corpus (13)

• Communication-Efficient Learning of Deep Networks from Decentralized Data
• Federated Learning: Strategies for Improving Communication Efficiency
• Comprehensive Privacy Analysis of Deep Learning: Passive and Active White-box Inference Attacks against Centralized and Federated Learning
• QSGD: Communication-Efficient SGD via Gradient Quantization and Encoding
• TernGrad: Ternary Gradients to Reduce Communication in Distributed Deep Learning
• Measuring the Effects of Non-Identical Data Distribution for Federated Visual Classification
• Byzantine-Robust Distributed Learning: Towards Optimal Statistical Rates
• Gradient Sparsification for Communication-Efficient Distributed Optimization
• Distributed Mean Estimation with Limited Communication
• FastSecAgg: Scalable Secure Aggregation for Privacy-Preserving Federated Learning
• Zeno: Distributed Stochastic Gradient Descent with Suspicion-based Fault-tolerance
• Dynamic Sampling and Selective Masking for Communication-Efficient Federated Learning
• ByGARS: Byzantine SGD with Arbitrary Number of Attackers

Cited by in corpus (4)

• Group Knowledge Transfer: Federated Learning of Large CNNs at the Edge
• FedGraphNN: A Federated Learning System and Benchmark for Graph Neural Networks
• Basil: A Fast and Byzantine-Resilient Approach for Decentralized Training
• Secure Aggregation for Buffered Asynchronous Federated Learning