Noiseless Privacy-Preserving Decentralized Learning
arXiv:2404.09536 · doi:10.56553/popets-2025-0043
Abstract
Decentralized learning (DL) enables collaborative learning without a server and without training data leaving the users' devices. However, the models shared in DL can still be used to infer training data. Conventional defenses such as differential privacy and secure aggregation fall short in effectively safeguarding user privacy in DL, either sacrificing model utility or efficiency. We introduce Shatter, a novel DL approach in which nodes create virtual nodes (VNs) to disseminate chunks of their full model on their behalf. This enhances privacy by (i) preventing attackers from collecting full models from other nodes, and (ii) hiding the identity of the original node that produced a given model chunk. We theoretically prove the convergence of Shatter and provide a formal analysis demonstrating how Shatter reduces the efficacy of attacks compared to when exchanging full models between nodes. We evaluate the convergence and attack resilience of Shatter with existing DL algorithms, with heterogeneous datasets, and against three standard privacy attacks. Our evaluation shows that Shatter not only renders these privacy attacks infeasible when each node operates 16 VNs but also exhibits a positive impact on model utility compared to standard DL. In summary, Shatter enhances the privacy of DL while maintaining the utility and efficiency of the model.
Accepted at PETS 2025
References in corpus (16)
- RAPPOR: Randomized Aggregatable Privacy-Preserving Ordinal Response
- Deep Gradient Compression: Reducing the Communication Bandwidth for Distributed Training
- Measuring the Effects of Non-Identical Data Distribution for Federated Visual Classification
- Decentralized Federated Learning: Fundamentals, State of the Art, Frameworks, Trends, and Challenges
- Asynchronous Stochastic Gradient Descent with Delay Compensation
- Gossip Learning with Linear Models on Fully Distributed Data
- ML-Doctor: Holistic Risk Assessment of Inference Attacks Against Machine Learning Models
- Shielding Federated Learning Systems against Inference Attacks with ARM TrustZone
- Decentralized Learning Made Easy with DecentralizePy
- Gradient Obfuscation Gives a False Sense of Security in Federated Learning
- MixNN: Protection of Federated Learning Against Inference Attacks by Mixing Neural Network Layers
- PRIVIC: A privacy-preserving method for incremental collection of location data
- Epidemic Learning: Boosting Decentralized Learning with Randomized Communication
- TEE-based decentralized recommender systems: The raw data sharing redemption
- Get More for Less in Decentralized Learning Systems
- Asynchronous SGD on Graphs: a Unified Framework for Asynchronous Decentralized and Federated Optimization