Privacy-preserving Collaborative Learning with Automatic Transformation Search

Abstract

Collaborative learning has gained great popularity due to its benefit of data privacy protection: participants can jointly train a Deep Learning model without sharing their training sets. However, recent works discovered that an adversary can fully recover the sensitive training samples from the shared gradients. Such reconstruction attacks pose severe threats to collaborative learning. Hence, effective mitigation solutions are urgently desired. In this paper, we propose to leverage data augmentation to defeat reconstruction attacks: by preprocessing sensitive images with carefully-selected transformation policies, it becomes infeasible for the adversary to extract any useful information from the corresponding gradients. We design a novel search method to automatically discover qualified policies. We adopt two new metrics to quantify the impacts of transformations on data privacy and model usability, which can significantly accelerate the search speed. Comprehensive evaluations demonstrate that the policies discovered by our method can defeat existing reconstruction attacks in collaborative learning, with high efficiency and negligible impact on the model performance.

17 pages, 16 figures, CVPR2021 oral

References in corpus (18)

• PyTorch: An Imperative Style, High-Performance Deep Learning Library
• Deep Learning with Differential Privacy
• Fashion-MNIST: a Novel Image Dataset for Benchmarking Machine Learning Algorithms
• Neural Architecture Search with Reinforcement Learning
• Comprehensive Privacy Analysis of Deep Learning: Passive and Active White-box Inference Attacks against Centralized and Federated Learning
• AutoAugment: Learning Augmentation Policies from Data
• iDLG: Improved Deep Leakage from Gradients
• Deep Models Under the GAN: Information Leakage from Collaborative Deep Learning
• A Survey on Neural Architecture Search
• EDA: Easy Data Augmentation Techniques for Boosting Performance on Text Classification Tasks
• A Framework for Evaluating Gradient Leakage Attacks in Federated Learning
• Neural Architecture Search without Training
• DeepSweep: An Evaluation Framework for Mitigating DNN Backdoor Attacks using Data Augmentation
• Scalable Differential Privacy with Certified Robustness in Adversarial Learning
• FenceBox: A Platform for Defeating Adversarial Examples with Data Augmentation Techniques
• Mitigating Advanced Adversarial Attacks with More Advanced Gradient Obfuscation Techniques
• Rethinking Privacy Preserving Deep Learning: How to Evaluate and Thwart Privacy Attacks
• Byzantine-resilient Decentralized Stochastic Gradient Descent