Federated Learning via Plurality Vote

Abstract

Federated learning allows collaborative workers to solve a machine learning problem while preserving data privacy. Recent studies have tackled various challenges in federated learning, but the joint optimization of communication overhead, learning reliability, and deployment efficiency is still an open problem. To this end, we propose a new scheme named federated learning via plurality vote (FedVote). In each communication round of FedVote, workers transmit binary or ternary weights to the server with low communication overhead. The model parameters are aggregated via weighted voting to enhance the resilience against Byzantine attacks. When deployed for inference, the model with binary or ternary weights is resource-friendly to edge devices. We show that our proposed method can reduce quantization error and converges faster compared with the methods directly quantizing the model updates.

References in corpus (12)

• Fashion-MNIST: a Novel Image Dataset for Benchmarking Machine Learning Algorithms
• Communication-Efficient Learning of Deep Networks from Decentralized Data
• Binarized Neural Networks
• Measuring the Effects of Non-Identical Data Distribution for Federated Visual Classification
• Byzantine-Robust Distributed Learning: Towards Optimal Statistical Rates
• Binary Neural Networks: A Survey
• Ensemble Distillation for Robust Model Fusion in Federated Learning
• FedPAQ: A Communication-Efficient Federated Learning Method with Periodic Averaging and Quantization
• HeteroFL: Computation and Communication Efficient Federated Learning for Heterogeneous Clients
• Cooperative SGD: A unified Framework for the Design and Analysis of Communication-Efficient SGD Algorithms
• Byzantine-Robust Federated Machine Learning through Adaptive Model Averaging
• Stochastic Sign Descent Methods: New Algorithms and Better Theory