Privacy-preserving Collaborative Learning with Automatic Transformation Search

Abstract

Collaborative learning has gained great popularity due to its benefit of data privacy protection: participants can jointly train a Deep Learning model without sharing their training sets. However, recent works discovered that an adversary can fully recover the sensitive training samples from the shared gradients. Such reconstruction attacks pose severe threats to collaborative learning. Hence, effective mitigation solutions are urgently desired. In this paper, we propose to leverage data augmentation to defeat reconstruction attacks: by preprocessing sensitive images with carefully-selected transformation policies, it becomes infeasible for the adversary to extract any useful information from the corresponding gradients. We design a novel search method to automatically discover qualified policies. We adopt two new metrics to quantify the impacts of transformations on data privacy and model usability, which can significantly accelerate the search speed. Comprehensive evaluations demonstrate that the policies discovered by our method can defeat existing reconstruction attacks in collaborative learning, with high efficiency and negligible impact on the model performance.

17 pages, 16 figures, CVPR2021 oral

References in corpus (9)

• PyTorch: An Imperative Style, High-Performance Deep Learning Library
• Fashion-MNIST: a Novel Image Dataset for Benchmarking Machine Learning Algorithms
• Neural Architecture Search with Reinforcement Learning
• iDLG: Improved Deep Leakage from Gradients
• A Survey on Neural Architecture Search
• A Framework for Evaluating Gradient Leakage Attacks in Federated Learning
• FenceBox: A Platform for Defeating Adversarial Examples with Data Augmentation Techniques
• Mitigating Advanced Adversarial Attacks with More Advanced Gradient Obfuscation Techniques
• Rethinking Privacy Preserving Deep Learning: How to Evaluate and Thwart Privacy Attacks