Asynchronous Federated Learning with Differential Privacy for Edge Intelligence

Abstract

Federated learning has been showing as a promising approach in paving the last mile of artificial intelligence, due to its great potential of solving the data isolation problem in large scale machine learning. Particularly, with consideration of the heterogeneity in practical edge computing systems, asynchronous edge-cloud collaboration based federated learning can further improve the learning efficiency by significantly reducing the straggler effect. Despite no raw data sharing, the open architecture and extensive collaborations of asynchronous federated learning (AFL) still give some malicious participants great opportunities to infer other parties' training data, thus leading to serious concerns of privacy. To achieve a rigorous privacy guarantee with high utility, we investigate to secure asynchronous edge-cloud collaborative federated learning with differential privacy, focusing on the impacts of differential privacy on model convergence of AFL. Formally, we give the first analysis on the model convergence of AFL under DP and propose a multi-stage adjustable private algorithm (MAPA) to improve the trade-off between model utility and privacy by dynamically adjusting both the noise scale and the learning rate. Through extensive simulations and real-world experiments with an edge-could testbed, we demonstrate that MAPA significantly improves both the model accuracy and convergence speed with sufficient privacy guarantee.

References in corpus (14)

• Auto-Encoding Variational Bayes
• Deep Learning with Differential Privacy
• Communication-Efficient Learning of Deep Networks from Decentralized Data
• Federated Learning: Strategies for Improving Communication Efficiency
• Federated Optimization: Distributed Machine Learning for On-Device Intelligence
• Convolutional Neural Network Architectures for Matching Natural Language Sentences
• Learning Differentially Private Recurrent Language Models
• Differentially Private Federated Learning: A Client Level Perspective
• Trunk-Branch Ensemble Convolutional Neural Networks for Video-based Face Recognition
• Distributed Delayed Stochastic Optimization
• Differentially Private Learning with Adaptive Clipping
• Asynchronous Coordinate Descent under More Realistic Assumptions
• Interpret Federated Learning with Shapley Values
• More Iterations per Second, Same Quality -- Why Asynchronous Algorithms may Drastically Outperform Traditional Ones

Cited by in corpus (3)

• Towards Efficient and Stable K-Asynchronous Federated Learning with Unbounded Stale Gradients on Non-IID Data
• Asynchronous Federated Learning on Heterogeneous Devices: A Survey
• A(DP)${}^2$SGD: Asynchronous Decentralized Parallel Stochastic Gradient Descent with Differential Privacy