Dubhe: Towards Data Unbiasedness with Homomorphic Encryption in Federated Learning Client Selection

Abstract

Federated learning (FL) is a distributed machine learning paradigm that allows clients to collaboratively train a model over their own local data. FL promises the privacy of clients and its security can be strengthened by cryptographic methods such as additively homomorphic encryption (HE). However, the efficiency of FL could seriously suffer from the statistical heterogeneity in both the data distribution discrepancy among clients and the global distribution skewness. We mathematically demonstrate the cause of performance degradation in FL and examine the performance of FL over various datasets. To tackle the statistical heterogeneity problem, we propose a pluggable system-level client selection method named Dubhe, which allows clients to proactively participate in training, meanwhile preserving their privacy with the assistance of HE. Experimental results show that Dubhe is comparable with the optimal greedy method on the classification accuracy, with negligible encryption and communication overhead.

arXiv comment10 pages

References in corpus (15)

• SMOTE: Synthetic Minority Over-sampling Technique
• Communication-Efficient Learning of Deep Networks from Decentralized Data
• Federated Learning with Non-IID Data
• A Survey on Federated Learning Systems: Vision, Hype and Reality for Data Privacy and Protection
• An Analysis of Deep Neural Network Models for Practical Applications
• Secure Federated Transfer Learning
• Tackling the Objective Inconsistency Problem in Heterogeneous Federated Optimization
• Differentially Private Federated Learning: A Client Level Perspective
• Federated Optimization in Heterogeneous Networks
• Private federated learning on vertically partitioned data via entity resolution and additively homomorphic encryption
• FedML: A Research Library and Benchmark for Federated Machine Learning
• LEAF: A Benchmark for Federated Settings
• Adaptive Federated Optimization
• Active Federated Learning
• Client Selection in Federated Learning: Convergence Analysis and Power-of-Choice Selection Strategies