Topology-aware Differential Privacy for Decentralized Image Classification

Abstract

In this paper, we design Top-DP, a novel solution to optimize the differential privacy protection of decentralized image classification systems. The key insight of our solution is to leverage the unique features of decentralized communication topologies to reduce the noise scale and improve the model usability. (1) We enhance the DP-SGD algorithm with this topology-aware noise reduction strategy, and integrate the time-aware noise decay technique. (2) We design two novel learning protocols (synchronous and asynchronous) to protect systems with different network connectivities and topologies. We formally analyze and prove the DP requirement of our proposed solutions. Experimental evaluations demonstrate that our solution achieves a better trade-off between usability and privacy than prior works. To the best of our knowledge, this is the first DP optimization work from the perspective of network topologies.

Accepted by TCSVT

References in corpus (7)

• Differentially Private Federated Learning: A Client Level Perspective
• Protection Against Reconstruction and Its Applications in Private Federated Learning
• Differential Privacy-enabled Federated Learning for Sensitive Health Data
• Stolen Memories: Leveraging Model Memorization for Calibrated White-Box Membership Inference
• Improving the Privacy and Accuracy of ADMM-Based Distributed Algorithms
• Weighted Distributed Differential Privacy ERM: Convex and Non-convex
• Heterogeneity-Aware Asynchronous Decentralized Training