Differentially Private Federated Learning via Inexact ADMM

Abstract

Differential privacy (DP) techniques can be applied to the federated learning model to protect data privacy against inference attacks to communication among the learning agents. The DP techniques, however, hinder achieving a greater learning performance while ensuring strong data privacy. In this paper we develop a DP inexact alternating direction method of multipliers algorithm that solves a sequence of subproblems with the objective perturbation by random noises generated from a Laplace distribution. We show that our algorithm provides $\bar{\epsilon }$-DP for every iteration, where $\bar{\epsilon }$ is a privacy parameter controlled by a user. Using MNIST and FEMNIST datasets for the image classification, we demonstrate that our algorithm reduces the testing error by at most $22\%$ compared with the existing DP algorithm, while achieving the same level of data privacy. The numerical experiment also shows that our algorithm converges faster than the existing algorithm.

arXiv comment20 pages, 6 figures

References in corpus (9)

• Deep Learning with Differential Privacy
• Communication-Efficient Learning of Deep Networks from Decentralized Data
• Deep EHR: A Survey of Recent Advances in Deep Learning Techniques for Electronic Health Record (EHR) Analysis
• Towards Deep Learning Models Resistant to Adversarial Attacks
• A Survey on Federated Learning Systems: Vision, Hype and Reality for Data Privacy and Protection
• Federated Optimization:Distributed Optimization Beyond the Datacenter
• Federated Optimization in Heterogeneous Networks
• LEAF: A Benchmark for Federated Settings
• Input Perturbation: A New Paradigm between Central and Local Differential Privacy

Cited by in corpus (1)

• Communication-Efficient ADMM-based Federated Learning