Differentially Private Federated Clustering over Non-IID Data

Abstract

In this paper, we investigate federated clustering (FedC) problem, that aims to accurately partition unlabeled data samples distributed over massive clients into finite clusters under the orchestration of a parameter server, meanwhile considering data privacy. Though it is an NP-hard optimization problem involving real variables denoting cluster centroids and binary variables denoting the cluster membership of each data sample, we judiciously reformulate the FedC problem into a non-convex optimization problem with only one convex constraint, accordingly yielding a soft clustering solution. Then a novel FedC algorithm using differential privacy (DP) technique, referred to as DP-FedC, is proposed in which partial clients participation and multiple local model updating steps are also considered. Furthermore, various attributes of the proposed DP-FedC are obtained through theoretical analyses of privacy protection and convergence rate, especially for the case of non-identically and independently distributed (non-i.i.d.) data, that ideally serve as the guidelines for the design of the proposed DP-FedC. Then some experimental results on two real datasets are provided to demonstrate the efficacy of the proposed DP-FedC together with its much superior performance over some state-of-the-art FedC algorithms, and the consistency with all the presented analytical results.

arXiv comment34 pages, 4 figures, 1 table

References in corpus (12)

• Deep Learning with Differential Privacy
• Federated Optimization: Distributed Machine Learning for On-Device Intelligence
• Federated Optimization:Distributed Optimization Beyond the Datacenter
• Secure Federated Matrix Factorization
• FedPD: A Federated Learning Framework with Optimal Rates and Adaptivity to Non-IID Data
• Robust Federated Learning in a Heterogeneous Environment
• Learning From Hidden Traits: Joint Factor Analysis and Latent Clustering
• On the Convergence of Clustered Federated Learning
• Towards Federated Clustering: A Federated Fuzzy $c$-Means Algorithm (FFCM)
• Federated Semi-Supervised Learning with Class Distribution Mismatch
• Secure Federated Clustering
• Federated Stochastic Primal-dual Learning with Differential Privacy

Cited by in corpus (2)

• One-Shot Federated Clustering of Non-Independent Completely Distributed Data
• Privacy-preserving Federated Primal-dual Learning for Non-convex and Non-smooth Problems with Model Sparsification