FedBoosting: Federated Learning with Gradient Protected Boosting for Text Recognition

Abstract

Typical machine learning approaches require centralized data for model training, which may not be possible where restrictions on data sharing are in place due to, for instance, privacy and gradient protection. The recently proposed Federated Learning (FL) framework allows learning a shared model collaboratively without data being centralized or shared among data owners. However, we show in this paper that the generalization ability of the joint model is poor on Non-Independent and Non-Identically Distributed (Non-IID) data, particularly when the Federated Averaging (FedAvg) strategy is used due to the weight divergence phenomenon. Hence, we propose a novel boosting algorithm for FL to address both the generalization and gradient leakage issues, as well as achieve faster convergence in gradient-based optimization. In addition, a secure gradient sharing protocol using Homomorphic Encryption (HE) and Differential Privacy (DP) is introduced to defend against gradient leakage attack and avoid pairwise encryption that is not scalable. We demonstrate the proposed Federated Boosting (FedBoosting) method achieves noticeable improvements in both prediction accuracy and run-time efficiency in a visual text recognition task on public benchmark.

The source code can be found at https://github.com/Rand2AI/FedBoosting

References in corpus (9)

• Federated Learning with Personalization Layers
• Ensemble Distillation for Robust Model Fusion in Federated Learning
• FedMD: Heterogenous Federated Learning via Model Distillation
• Private federated learning on vertically partitioned data via entity resolution and additively homomorphic encryption
• iDLG: Improved Deep Leakage from Gradients
• A generic framework for privacy preserving deep learning
• Decentralized Federated Learning via Mutual Knowledge Transfer
• GRNN: Generative Regression Neural Network -- A Data Leakage Attack for Federated Learning
• FedOCR: Communication-Efficient Federated Learning for Scene Text Recognition

Cited by in corpus (1)

• An Element-Wise Weights Aggregation Method for Federated Learning